zeripath bc4764ffc6 Detect truncated utf-8 characters at the end of content as still representing utf-8 (#19773) ...

Our character detection algorithm can potentially incorrectly detect utf-8 as iso-8859-x
if there is a truncated character at the end of the partially read file.

This PR changes the detection algorithm to truncated utf8 characters at the end of the
buffer.

Fix #19743

Signed-off-by: Andrew Thornton <art27@cantab.net>

2022-05-21 14:06:24 +01:00

..

charset_test.go

Detect truncated utf-8 characters at the end of content as still representing utf-8 (#19773)

2022-05-21 14:06:24 +01:00

charset.go

Detect truncated utf-8 characters at the end of content as still representing utf-8 (#19773)

2022-05-21 14:06:24 +01:00

escape_test.go

Don't treat BOM escape sequence as hidden character. (#18909)

2022-02-26 16:48:23 +00:00

escape.go

Don't treat BOM escape sequence as hidden character. (#18909)

2022-02-26 16:48:23 +00:00