Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
summaryrefslogtreecommitdiff
path: root/Objects/unicodeobject.c
Commit message (Collapse)AuthorAgeFilesLines
* [3.7] bpo-38409: Fix grammar in str.strip() docstring (GH-16682) (GH-16686)Zachary Ware2019-10-091-2/+2
| | | (cherry picked from commit 09895c27cd8ff60563a794016e8c099bc897cc74)
* bpo-36311: Fixes decoding multibyte characters around chunk boundaries and ↵Miss Islington (bot)2019-08-211-6/+10
| | | | | | | improves decoding performance (GH-15083) (cherry picked from commit 7ebdda0dbee7df6f0c945a7e1e623e47676e112d) Co-authored-by: Steve Dower <steve.dower@python.org>
* [3.7] bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304) ↵Miss Islington (bot)2019-06-251-3/+7
| | | | | | | | | | | | | (GH-14369) * bpo-24214: Fixed the UTF-8 and UTF-16 incremental decoders. (GH-14304) * The UTF-8 incremental decoders fails now fast if encounter a sequence that can't be handled by the error handler. * The UTF-16 incremental decoders with the surrogatepass error handler decodes now a lone low surrogate with final=False. (cherry picked from commit 894263ba80af4b7733c2df95b527e96953922656) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-36946: Fix possible signed integer overflow when handling slices. (GH-13375)Miss Islington (bot)2019-05-171-1/+2
| | | | | | | | | | The final addition (cur += step) may overflow, so use size_t for "cur". "cur" is always positive (even for negative steps), so it is safe to use size_t here. Co-Authored-By: Martin Panter <vadmium+py@gmail.com> (cherry picked from commit 14514d9084a40f599c57da853a305aa264562a43) Co-authored-by: Zackery Spytz <zspytz@gmail.com>
* bpo-24214: Fixed the UTF-8 incremental decoder. (GH-12603) (GH-12627)Miss Islington (bot)2019-03-301-0/+3
| | | | | | | The bug occurred when the encoded surrogate character is passed to the incremental decoder in two chunks. (cherry picked from commit 7a465cb5ee7e298cae626ace1fc3e7d97df79f2e) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-36312: Fix decoders for some code pages. (GH-12369)Miss Islington (bot)2019-03-201-5/+16
| | | | | (cherry picked from commit c1e2c288f41cdc1c6e6e09d9a5277a58232ceb03) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-35552: Fix reading past the end in PyUnicode_FromFormat() and ↵Miss Islington (bot)2019-01-121-3/+9
| | | | | | | | | PyBytes_FromFormat(). (GH-11276) Format characters "%s" and "%V" in PyUnicode_FromFormat() and "%s" in PyBytes_FromFormat() no longer read memory past the limit if precision is specified. (cherry picked from commit d586ccb04f79863c819b212ec5b9d873964078e4) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-35560: Remove assertion from format(float, "n") (GH-11288)Miss Islington (bot)2019-01-071-1/+1
| | | | | | | | Fix an assertion error in format() in debug build for floating point formatting with "n" format, zero padding and small width. Release build is not impacted. Patch by Karthikeyan Singaravelan. (cherry picked from commit 3f7983a25a3d19779283c707fbdd5bc91b1587ef) Co-authored-by: Xtreak <tir.karthi@gmail.com>
* bpo-35372: Fix the code page decoder for input > 2 GiB. (GH-10848)Miss Islington (bot)2018-12-031-5/+4
| | | | | (cherry picked from commit 4013c179117754b039957db4730880bf3285919d) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-34523, bpo-35322: Fix unicode_encode_locale() (GH-10759) (GH-10761)Victor Stinner2018-11-281-7/+5
| | | | | | | | Fix memory leak in PyUnicode_EncodeLocale() and PyUnicode_EncodeFSDefault() on error handling. Fix unicode_encode_locale() error handling. (cherry picked from commit bde9d6bbb46ca59bcee5d5060adaa33c3ffee3a6)
* bpo-33954: Rewrite FILL() macro of unicodeobject.c (GH-10738)Victor Stinner2018-11-271-9/+17
| | | | | | | Copy code from master: add assertions on start and value, replace 'i' iterator with 'end' pointer for the loop stop condition. _PyUnicode_FastFill(): fix type of 'data', it must not be constant, since data is modified by FILL().
* bpo-33954: Fix _PyUnicode_InsertThousandsGrouping() (GH-10623) (GH-10718)Victor Stinner2018-11-261-92/+152
| | | | | | | | | | Fix str.format(), float.__format__() and complex.__format__() methods for non-ASCII decimal point when using the "n" formatter. Rewrite _PyUnicode_InsertThousandsGrouping(): it now requires a _PyUnicodeWriter object for the buffer and a Python str object for digits. (cherry picked from commit 59423e3ddd736387cef8f7632c71954c1859bed0)
* [3.7] bpo-35214: Fix OOB memory access in unicode escape parser (GH-10506) ↵Miss Islington (bot)2018-11-131-1/+1
| | | | | | | | | | | | | | | (GH-10522) Discovered using clang's MemorySanitizer when it ran python3's test_fstring test_misformed_unicode_character_name. An msan build will fail by simply executing: ./python -c 'u"\N"' (cherry picked from commit 746b2d35ea47005054ed774fecaed64fab803d7d) Co-authored-by: Gregory P. Smith <greg@krypto.org> https://bugs.python.org/issue35214
* bpo-34435: Add missing NULL check to unicode_encode_ucs1(). (GH-8823)Miss Islington (bot)2018-08-191-2/+3
| | | | | | Reported by Svace static analyzer. (cherry picked from commit 74a307d48ef8b278c4629ca0ef2139be1c9a34e6) Co-authored-by: Alexey Izbyshev <izbyshev@users.noreply.github.com>
* bpo-34087: Fix buffer overflow in int(s) and similar functions (GH-8274)Miss Islington (bot)2018-07-131-0/+2
| | | | | | | | | `_PyUnicode_TransformDecimalAndSpaceToASCII()` missed trailing NUL char. It caused buffer overflow in `_Py_string_to_number_with_underscores()`. This bug is introduced in 9b6c60cb. (cherry picked from commit 16dfca4d829e45f36e71bf43f83226659ce49315) Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
* bpo-32827: Fix usage of _PyUnicodeWriter_Prepare() in decoding errors ↵Miss Islington (bot)2018-02-131-7/+3
| | | | | | | | handler. (GH-5636) (GH-5650) (cherry picked from commit b7e2d67f7c035f09c921ca4e7a36529cd502ccf7) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-32583: Fix possible crashing in builtin Unicode decoders (#5325)Xiang Zhang2018-01-311-2/+20
| | | | | When using customized decode error handlers, it is possible for builtin decoders to write out-of-bounds and then crash.
* Fix wrong assert in unicodeobject (GH-5340)INADA Naoki2018-01-281-1/+1
|
* bpo-32677: Add .isascii() to str, bytes and bytearray (GH-5342)INADA Naoki2018-01-271-0/+20
|
* bpo-29240: Fix locale encodings in UTF-8 Mode (#5170)Victor Stinner2018-01-151-331/+144
| | | | | | | | | | | | | | | | | | | | | | | | | | | Modify locale.localeconv(), time.tzname, os.strerror() and other functions to ignore the UTF-8 Mode: always use the current locale encoding. Changes: * Add _Py_DecodeLocaleEx() and _Py_EncodeLocaleEx(). On decoding or encoding error, they return the position of the error and an error message which are used to raise Unicode errors in PyUnicode_DecodeLocale() and PyUnicode_EncodeLocale(). * Replace _Py_DecodeCurrentLocale() with _Py_DecodeLocaleEx(). * PyUnicode_DecodeLocale() now uses _Py_DecodeLocaleEx() for all cases, especially for the strict error handler. * Add _Py_DecodeUTF8Ex(): return more information on decoding error and supports the strict error handler. * Rename _Py_EncodeUTF8_surrogateescape() to _Py_EncodeUTF8Ex(). * Replace _Py_EncodeCurrentLocale() with _Py_EncodeLocaleEx(). * Ignore the UTF-8 mode to encode/decode localeconv(), strerror() and time zone name. * Remove PyUnicode_DecodeLocale(), PyUnicode_DecodeLocaleAndSize() and PyUnicode_EncodeLocale() now ignore the UTF-8 mode: always use the "current" locale. * Remove _PyUnicode_DecodeCurrentLocale(), _PyUnicode_DecodeCurrentLocaleAndSize() and _PyUnicode_EncodeCurrentLocale().
* bpo-29240: Ignore UTF-8 Mode in time module (#5148)Victor Stinner2018-01-111-0/+6
| | | | | | time.strftime() must use the current LC_CTYPE encoding, not UTF-8 if the UTF-8 mode is enabled. Add _PyUnicode_DecodeCurrentLocale() function.
* bpo-29240: readline now ignores the UTF-8 Mode (#5145)Victor Stinner2018-01-101-10/+52
| | | | | | | | | | | | Add new fuctions ignoring the UTF-8 mode: * _Py_DecodeCurrentLocale() * _Py_EncodeCurrentLocale() * _PyUnicode_DecodeCurrentLocaleAndSize() * _PyUnicode_EncodeCurrentLocale() Modify the readline module to use these functions. Re-enable test_readline.test_nonascii().
* bpo-32030: Add _Py_EncodeLocaleRaw() (#4961)Victor Stinner2017-12-211-4/+21
| | | | | | | | | | | | Replace Py_EncodeLocale() with _Py_EncodeLocaleRaw() in: * _Py_wfopen() * _Py_wreadlink() * _Py_wrealpath() * _Py_wstat() * pymain_open_filename() These functions are called early during Python intialization, only the RAW memory allocator must be used.
* bpo-32030: Add _Py_EncodeUTF8_surrogateescape() (#4960)Victor Stinner2017-12-211-0/+89
| | | | | Py_EncodeLocale() now uses _Py_EncodeUTF8_surrogateescape(), instead of using temporary unicode and bytes objects. So Py_EncodeLocale() doesn't use the Python C API anymore.
* bpo-32240: Add the const qualifier to declarations of PyObject* array ↵Serhiy Storchaka2017-12-151-1/+1
| | | | arguments. (#4746)
* bpo-29240: PEP 540: Add a new UTF-8 Mode (#855)Victor Stinner2017-12-131-9/+21
| | | | | | | | | | | | | | | | | | | | | | * Add -X utf8 command line option, PYTHONUTF8 environment variable and a new sys.flags.utf8_mode flag. * If the LC_CTYPE locale is "C" at startup: enable automatically the UTF-8 mode. * Add _winapi.GetACP(). encodings._alias_mbcs() now calls _winapi.GetACP() to get the ANSI code page * locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8 mode. As a side effect, open() now uses the UTF-8 encoding by default in this mode. * Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding in the UTF-8 Mode. * Update subprocess._args_from_interpreter_flags() to handle -X utf8 * Skip some tests relying on the current locale if the UTF-8 mode is enabled. * Add test_utf8mode.py. * _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to return also the length (number of wide characters). * pymain_get_global_config() and pymain_set_global_config() now always copy flag values, rather than only copying if the new value is greater than the old value.
* bpo-31979: Remove unused align_maxchar() function (#4527)Victor Stinner2017-11-231-13/+0
|
* bpo-31979: Simplify transforming decimals to ASCII (#4336)Serhiy Storchaka2017-11-131-104/+32
| | | | | in int(), float() and complex() parsers. This also speeds up parsing non-ASCII numbers by around 20%.
* Add the const qualifier to "char *" variables that refer to literal strings. ↵Serhiy Storchaka2017-11-111-3/+3
| | | | (#4370)
* bpo-23699: Use a macro to reduce boilerplate code in rich comparison ↵stratakis2017-11-021-30/+4
| | | | functions (GH-793)
* bpo-20047: Make bytearray methods partition() and rpartition() rejecting (#4158)Serhiy Storchaka2017-10-291-2/+2
| | | separators that are not bytes-like objects.
* bpo-31825: Fixed OverflowError in the 'unicode-escape' codec (#4058)Serhiy Storchaka2017-10-201-1/+1
| | | | and in codecs.escape_decode() when decode an escaped non-ascii byte.
* bpo-31338 (#3374)Barry Warsaw2017-09-141-50/+34
| | | | | | | * Add Py_UNREACHABLE() as an alias to abort(). * Use Py_UNREACHABLE() instead of assert(0) * Convert more unreachable code to use Py_UNREACHABLE() * Document Py_UNREACHABLE() and a few other macros.
* bpo-31393: Fix the use of PyUnicode_READY(). (#3451)Serhiy Storchaka2017-09-081-2/+8
|
* bpo-30860: Consolidate stateful runtime globals. (#3397)Eric Snow2017-09-071-0/+1
| | | | | | | * group the (stateful) runtime globals into various topical structs * consolidate the topical structs under a single top-level _PyRuntimeState struct * add a check-c-globals.py script that helps identify runtime globals Other globals are excluded (see globals.txt and check-c-globals.py).
* bpo-30923: Silence fall-through warnings included in -Wextra since gcc-7.0. ↵Stefan Krah2017-08-211-2/+3
| | | | (#3157)
* bpo-22207: Add checks for possible integer overflows in unicodeobject.c. (#2623)Serhiy Storchaka2017-07-111-6/+12
| | | Based on patch by Victor Stinner.
* [security] bpo-13617: Reject embedded null characters in wchar* strings. (#2302)Serhiy Storchaka2017-06-281-0/+14
| | | | | | | Based on patch by Victor Stinner. Add private C API function _PyUnicode_AsUnicode() which is similar to PyUnicode_AsUnicode(), but checks for null characters.
* bpo-30708: Check for null characters in PyUnicode_AsWideCharString(). (#2285)Serhiy Storchaka2017-06-271-27/+22
| | | | | Raise a ValueError if the second argument is NULL and the wchar_t\* string contains null characters.
* bpo-29802: Fix reference counting in module-level struct functions (#1213)Serhiy Storchaka2017-04-201-0/+1
| | | | when pass arguments of wrong type.
* Expand the PySlice_GetIndicesEx macro. (#1023)Serhiy Storchaka2017-04-081-2/+3
|
* bpo-29549: Fixes docstring for str.index (#256)Lisa Roach2017-04-041-2/+10
| | | | | | | | | | | | | | * Updates B.index documentation. * Updates str.index documentation, makes it Argument Clinic compatible. * Removes ArgumentClinic code. * Finishes string.index documentation. * Updates string.rindex documentation. * Documents B.rindex.
* bpo-29865: Use PyXXX_GET_SIZE macros rather than Py_SIZE for concrete types. ↵Serhiy Storchaka2017-03-211-1/+2
| | | | (#748)
* bpo-29116: Improve error message for concatenating str with non-str. (#710)Serhiy Storchaka2017-03-191-1/+10
|
* bpo-24037: Add Argument Clinic converter `bool(accept={int})`. (#485)Serhiy Storchaka2017-03-121-2/+2
|
* Use Py_RETURN_FALSE/Py_RETURN_TRUE rather than ↵Serhiy Storchaka2017-03-081-25/+25
| | | | PyBool_FromLong(0)/PyBool_FromLong(1). (#567)
* bpo-29568: Disable any characters between two percents for escaped percent ↵Serhiy Storchaka2017-03-081-7/+8
| | | | "%%" in the format string for classic string formatting. (GH-513)
* Fix grammar in doc string, RST markupMartin Panter2017-01-241-2/+2
|
* Issue #28999: Use Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE whereverSerhiy Storchaka2017-01-231-3/+1
| | | | possible. Patch is writen with Coccinelle.
* Issue #28769: The result of PyUnicode_AsUTF8AndSize() and PyUnicode_AsUTF8()Serhiy Storchaka2017-01-221-2/+2
| | | | is now of type "const char *" rather of "char *".