Bump json from 2.20.0 to 2.21.2 - #1031
Open
dependabot[bot] wants to merge 1 commit into
Open
Conversation
Bumps [json](https://github.com/ruby/json) from 2.20.0 to 2.21.2. - [Release notes](https://github.com/ruby/json/releases) - [Changelog](https://github.com/ruby/json/blob/master/CHANGES.md) - [Commits](ruby/json@v2.20.0...v2.21.2) --- updated-dependencies: - dependency-name: json dependency-version: 2.21.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com>
Contributor
4 similar comments
Contributor
Contributor
Contributor
Contributor
Contributor
gem compare json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT require_paths:
2.20.0: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.20.0", "lib"]
2.21.2: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.21.2", "lib"]
DIFFERENT rubygems_version:
2.20.0: 4.0.12
2.21.2: 4.0.16
DIFFERENT version:
2.20.0: 2.20.0
2.21.2: 2.21.2
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md +16/-0
README.md +5/-5
ext/json/ext/generator/generator.c +72/-3
ext/json/ext/json.h +4/-0
ext/json/ext/parser/extconf.rb +6/-16
ext/json/ext/parser/parser.c +140/-43
lib/json.rb +18/-3
lib/json/common.rb +9/-0
lib/json/ext.rb +26/-0
lib/json/ext/generator/state.rb +1/-0
lib/json/truffle_ruby/generator.rb +36/-1
lib/json/version.rb +1/-1
DIFFERENT extra_rdoc_files:
2.20.0->2.21.2:
* Changed:
README.md +5/-5 |
2 similar comments
Contributor
gem compare json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT require_paths:
2.20.0: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.20.0", "lib"]
2.21.2: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.21.2", "lib"]
DIFFERENT rubygems_version:
2.20.0: 4.0.12
2.21.2: 4.0.16
DIFFERENT version:
2.20.0: 2.20.0
2.21.2: 2.21.2
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md +16/-0
README.md +5/-5
ext/json/ext/generator/generator.c +72/-3
ext/json/ext/json.h +4/-0
ext/json/ext/parser/extconf.rb +6/-16
ext/json/ext/parser/parser.c +140/-43
lib/json.rb +18/-3
lib/json/common.rb +9/-0
lib/json/ext.rb +26/-0
lib/json/ext/generator/state.rb +1/-0
lib/json/truffle_ruby/generator.rb +36/-1
lib/json/version.rb +1/-1
DIFFERENT extra_rdoc_files:
2.20.0->2.21.2:
* Changed:
README.md +5/-5 |
Contributor
gem compare json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT require_paths:
2.20.0: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.20.0", "lib"]
2.21.2: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.21.2", "lib"]
DIFFERENT rubygems_version:
2.20.0: 4.0.12
2.21.2: 4.0.16
DIFFERENT version:
2.20.0: 2.20.0
2.21.2: 2.21.2
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md +16/-0
README.md +5/-5
ext/json/ext/generator/generator.c +72/-3
ext/json/ext/json.h +4/-0
ext/json/ext/parser/extconf.rb +6/-16
ext/json/ext/parser/parser.c +140/-43
lib/json.rb +18/-3
lib/json/common.rb +9/-0
lib/json/ext.rb +26/-0
lib/json/ext/generator/state.rb +1/-0
lib/json/truffle_ruby/generator.rb +36/-1
lib/json/version.rb +1/-1
DIFFERENT extra_rdoc_files:
2.20.0->2.21.2:
* Changed:
README.md +5/-5 |
Contributor
gem compare --diff json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md
--- /tmp/d20260809-476-31vay6/json-2.20.0/CHANGES.md 2026-08-09 11:50:38.655698823 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/CHANGES.md 2026-08-09 11:50:38.665698946 +0000
@@ -4,0 +5,16 @@
+### 2026-07-31 (2.21.2)
+
+* Fix a use-after-free bug in `JSON::ResumableParser`. [GHSA-9hj4-r449-hfvc].
+
+### 2026-07-13 (2.21.1)
+
+* Fix a compilation issue on Window and Microsoft Visual C++.
+
+### 2026-07-12 (2.21.0)
+
+* `JSON.generate` now accept a `sort_keys` option, which takes either a boolean or a block.
+* Added `#empty?` and `#partial_value?` methods on `JSON::ResumableParser`.
+* Numerous correctness and performance fixes for `JSON::ResumableParser`.
+* Avoid triggering Ruby's `float out of range` warning when parsing out of range numbers.
+* Declare C types with Ruby 4.1 `RUBY_TYPED_THREAD_SAFE_FREE`.
+
README.md
--- /tmp/d20260809-476-31vay6/json-2.20.0/README.md 2026-08-09 11:50:38.655698823 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/README.md 2026-08-09 11:50:38.665698946 +0000
@@ -88 +88 @@
-JSON.generate(Position.new(1, 2)) # => Position not allowed in JSON (JSON::GeneratorError)
+JSON.generate(Position.new(1, 2), strict: true) # => Position not allowed in JSON (JSON::GeneratorError)
@@ -120 +120 @@
-coder = JSON::Combining.new do |object, is_object_key|
+coder = JSON::Coder.new do |object, is_object_key|
@@ -123,2 +123,2 @@
- if !string.valid_encoding? || string.encoding != Encoding::UTF_8
- Base64.encode64(string)
+ if !object.valid_encoding? || object.encoding != Encoding::UTF_8
+ Base64.encode64(object)
@@ -126 +126 @@
- string
+ object
ext/json/ext/generator/generator.c
--- /tmp/d20260809-476-31vay6/json-2.20.0/ext/json/ext/generator/generator.c 2026-08-09 11:50:38.656698835 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/ext/json/ext/generator/generator.c 2026-08-09 11:50:38.666698958 +0000
@@ -36,0 +37 @@
+ VALUE sort_keys;
@@ -39 +40 @@
-static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8;
+static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8, default_sort_keys_proc;
@@ -43 +44 @@
- sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json;
+ sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json, sym_sort_keys;
@@ -711,0 +713 @@
+ rb_gc_mark_movable(state->sort_keys);
@@ -722,0 +725 @@
+ state->sort_keys = rb_gc_location(state->sort_keys);
@@ -742 +745 @@
- .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -771,0 +775 @@
+ RB_OBJ_WRITTEN(vstate, Qundef, state->sort_keys);
@@ -1052,0 +1057,5 @@
+ if (RB_UNLIKELY(data->state->sort_keys)) {
+ obj = rb_proc_call_with_block(data->state->sort_keys, 1, &obj, Qnil);
+ Check_Type(obj, T_HASH);
+ }
+
@@ -1378,0 +1388 @@
+ RB_OBJ_WRITTEN(obj, Qundef, objState->sort_keys);
@@ -1724,0 +1735,49 @@
+static VALUE cState_set_default_sort_keys_proc(VALUE self, VALUE proc)
+{
+ if (!rb_obj_is_proc(proc)) {
+ rb_raise(rb_eTypeError, "sort_key_proc must be a Proc");
+ }
+ return default_sort_keys_proc = proc;
+}
+
+static VALUE normalize_sort_keys(VALUE value)
+{
+ if (rb_obj_is_proc(value)) {
+ return value;
+ } else if (value == Qtrue) {
+ return default_sort_keys_proc;
+ } else if (RTEST(value)) {
+ rb_raise(rb_eTypeError, "The `sort_keys` argument must be a boolean or a Proc");
+ } else {
+ return Qfalse;
+ }
+}
+
+/*
+ * call-seq: sort_keys
+ *
+ * Get the value of sort_keys.
+ */
+static VALUE cState_sort_keys_p(VALUE self)
+{
+ GET_STATE(self);
+ return state->sort_keys;
+}
+
+/*
+ * call-seq: sort_keys=(value)
+ *
+ * value is a boolean or a proc. If the value is the boolean true, object keys
+ * will be sorted lexicographically in ascending order.
+ *
+ * If the value is a proc, it receives the entire Hash and must return a Hash
+ * with its pairs in the desired order, allowing for arbitrary sorting.
+ */
+static VALUE cState_sort_keys_set(VALUE self, VALUE value)
+{
+ rb_check_frozen(self);
+ GET_STATE(self);
+ RB_OBJ_WRITE(self, &state->sort_keys, normalize_sort_keys(value));
+ return Qnil;
+}
+
@@ -1834,0 +1894,3 @@
+ else if (key == sym_sort_keys) {
+ state_write_value(data, &state->sort_keys, normalize_sort_keys(val));
+ }
@@ -1911,0 +1974,2 @@
+ rb_global_variable(&default_sort_keys_proc);
+
@@ -1920,0 +1985,2 @@
+ rb_define_singleton_method(cState, "default_sort_keys_proc=", cState_set_default_sort_keys_proc, 1);
+
@@ -1959,0 +2026,2 @@
+ rb_define_method(cState, "sort_keys", cState_sort_keys_p, 0);
+ rb_define_method(cState, "sort_keys=", cState_sort_keys_set, 1);
@@ -1988,0 +2057 @@
+ sym_sort_keys = ID2SYM(rb_intern("sort_keys"));
ext/json/ext/json.h
--- /tmp/d20260809-476-31vay6/json-2.20.0/ext/json/ext/json.h 2026-08-09 11:50:38.656698835 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/ext/json/ext/json.h 2026-08-09 11:50:38.667698971 +0000
@@ -33,0 +34,4 @@
+#ifndef RUBY_TYPED_THREAD_SAFE_FREE
+#define RUBY_TYPED_THREAD_SAFE_FREE RUBY_TYPED_FREE_IMMEDIATELY
+#endif
+
ext/json/ext/parser/extconf.rb
--- /tmp/d20260809-476-31vay6/json-2.20.0/ext/json/ext/parser/extconf.rb 2026-08-09 11:50:38.656698835 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/ext/json/ext/parser/extconf.rb 2026-08-09 11:50:38.799700593 +0000
@@ -18,16 +17,0 @@
-def have_builtin_func(name, check_expr, opt = "", &b)
- checking_for checking_message(name.funcall_style, nil, opt) do
- if try_compile(<<SRC, opt, &b)
-int foo;
-int main() { #{check_expr}; return 0; }
-SRC
- $defs.push(format("-DHAVE_BUILTIN_%s", name.tr_cpp))
- true
- else
- false
- end
- end
-end
-
-have_builtin_func("__builtin_clzll", "__builtin_clzll(0)")
-
@@ -47,0 +32,6 @@
+
+# Disable function outlining on clang to prevent some of the repeated instructions
+# in json_eat_whitespace being outlined into function calls.
+# Note: This verifies '-mno-outline' is accepted as a valid compiler flag
+# and will not pass it if unsupported.
+append_cflags("-mno-outline")
ext/json/ext/parser/parser.c
--- /tmp/d20260809-476-31vay6/json-2.20.0/ext/json/ext/parser/parser.c 2026-08-09 11:50:38.657698848 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/ext/json/ext/parser/parser.c 2026-08-09 11:50:38.800700605 +0000
@@ -8 +8,4 @@
-static ID i_new, i_try_convert, i_uminus, i_encode, i_at_line, i_at_column;
+static ID i_new, i_try_convert, i_encode, i_at_line, i_at_column;
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
+static ID i_uminus;
+#endif
@@ -317 +320 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -514 +517 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
@@ -588,0 +592,2 @@
+ JSON_ASSERT(!state->parser);
+ JSON_ASSERT(state->cursor);
@@ -621,4 +626,8 @@
- long line, column;
- cursor_position(state, &line, &column);
-
- VALUE warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ VALUE warning;
+ if (state->parser) { // line and columns can't be accurate in resumable
+ warning = rb_utf8_str_new_cstr(message);
+ } else {
+ long line, column;
+ cursor_position(state, &line, &column);
+ warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ }
@@ -768 +777 @@
-json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config)
+json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config, const char *resume_pos)
@@ -774,0 +784,5 @@
+ // An incomplete comment suspends a resumable parse by rewinding the cursor
+ // and throwing. Callers that already consumed a token not yet committed to
+ // the frame stack pass resume_pos so the rewind re-reads that token too.
+ // Non-resumable error positions keep pointing at the comment either way.
+ const char *rewind_pos = (state->parser && resume_pos) ? resume_pos : start;
@@ -779,2 +793,9 @@
- state->cursor = memchr(state->cursor, '\n', state->end - state->cursor);
- if (!state->cursor) {
+ const char *newline = memchr(state->cursor, '\n', state->end - state->cursor);
+ if (!newline) {
+ // state->parser marks resumable mode, where the buffer end is only a
+ // chunk boundary: the terminating newline may still arrive, so leave
+ // the comment unterminated instead of consuming to end as a one-shot
+ // parse would.
+ if (state->parser) {
+ raise_eos_error_at("unterminated comment, expected end of line", state, rewind_pos);
+ }
@@ -783 +804 @@
- state->cursor++;
+ state->cursor = newline + 1;
@@ -793 +814 @@
- raise_eos_error_at("unterminated comment, expected closing '*/'", state, start);
+ raise_eos_error_at("unterminated comment, expected closing '*/'", state, rewind_pos);
@@ -805 +826 @@
- raise_parse_error_at("unexpected token %s", state, start, eos(state));
+ raise_parse_error_at("unexpected token %s", state, eos(state) ? rewind_pos : start, eos(state));
@@ -816 +837 @@
-json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+json_eat_whitespace_resume_at(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments, const char *resume_pos)
@@ -851 +872 @@
- json_eat_comments(state, config);
+ json_eat_comments(state, config, resume_pos);
@@ -859,0 +881,6 @@
+ALWAYS_INLINE(static) void
+json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+{
+ json_eat_whitespace_resume_at(state, config, include_comments, NULL);
+}
+
@@ -1132,0 +1160,7 @@
+ // If the value is so small that it definitely underflows to 0.0, return early
+ // to avoid triggering a "Float out of range" warning from rb_cstr_to_dbl.
+ // When mantissa_digits + exponent < -324, value < 10^(-324) < DBL_TRUE_MIN/2,
+ // so it rounds to 0 in IEEE 754 round-to-nearest.
+ if (RB_UNLIKELY(mantissa_digits + exponent < -324)) {
+ return rb_float_new(negative ? -0.0 : 0.0);
+ }
@@ -1432 +1466 @@
-static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start)
+static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start, bool resumable)
@@ -1488,0 +1523,10 @@
+ // A number touching the end of the buffer may still grow in a later chunk,
+ // so the caller will rewind and wait. Decoding it now would build a value
+ // -- for a long run of digits, an expensive bignum -- only to discard it,
+ // and repeating that on every resumed chunk is quadratic in the number's
+ // length. The digit scan above already advanced the cursor, which is all
+ // the caller needs to detect the incomplete number.
+ if (RB_UNLIKELY(resumable && eos(state))) {
+ return Qundef;
+ }
+
@@ -1565,0 +1610,7 @@
+ // A trailing comma lands us here expecting an element but finding the
+ // closing bracket; hand off to ARRAY_COMMA to close. An empty array
+ // closes inline at '[', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && frame->type == JSON_FRAME_ARRAY && peek(state) == ']') {
+ goto JSON_PHASE_ARRAY_COMMA;
+ }
+
@@ -1606 +1657 @@
- value = json_parse_number(state, config, true, value_start);
+ value = json_parse_number(state, config, true, value_start, resumable);
@@ -1629 +1680 @@
- value = json_parse_number(state, config, false, value_start);
+ value = json_parse_number(state, config, false, value_start, resumable);
@@ -1661 +1712,3 @@
- json_eat_whitespace(state, config, true);
+ // The '[' is consumed but its frame is only pushed below, so a
+ // comment suspending here must resume from the bracket.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1690 +1743,2 @@
- json_eat_whitespace(state, config, true);
+ // Same as '[': the frame is only pushed below.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1747,0 +1802,7 @@
+ // A trailing comma lands us here expecting a key but finding the closing
+ // brace; hand off to OBJECT_COMMA to close. An empty object closes inline
+ // at '{', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && peek(state) == '}') {
+ goto JSON_PHASE_OBJECT_COMMA;
+ }
+
@@ -1810,7 +1871,4 @@
- if (config->allow_trailing_comma) {
- json_eat_whitespace(state, config, true);
- if (peek(state) == ']') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_ARRAY_COMMA;
- }
- }
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_VALUE once the ']' is in the buffer.
@@ -1855,9 +1913,4 @@
- json_eat_whitespace(state, config, true);
-
- if (config->allow_trailing_comma) {
- if (peek(state) == '}') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_OBJECT_COMMA;
- }
- }
-
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_OBJECT_KEY once the '}' is in the buffer.
@@ -2183 +2236 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -2268 +2321 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -2521,0 +2575 @@
+ parser->state.start = parser->state.cursor = parser->state.end = 0;
@@ -2618 +2672,6 @@
- // Copy the value stack as we need to mutate it.
+ // Copy the value stack as we need to mutate it. The collapse loop folds each
+ // open container by popping its entries and pushing the single result, so a
+ // parent always reclaims its child's slot; head exceeds its live size by at
+ // most one, either for the missing-value placeholder pushed below or for the
+ // result of folding an empty innermost container. That one spare slot keeps
+ // rvalue_stack_push from growing (reallocating) this ALLOCV buffer.
@@ -2620,3 +2679,3 @@
- parser.value_stack.capa = (capa + missing_object_value);
- VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, capa + missing_object_value);
- MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, parser.value_stack.capa);
+ parser.value_stack.capa = capa + 1;
+ VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, parser.value_stack.capa);
+ MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, capa);
@@ -2715 +2774 @@
- * call-seq: value? -> true or false
+ * call-seq: eos? -> true or false
@@ -2725,0 +2785,35 @@
+ * call-seq: partial_value? -> true or false
+ *
+ * Returns whether a document is currently under construction: an unclosed
+ * container, a key awaiting its value, etc.
+ *
+ * It answers the same question as <tt>!partial_value.nil?</tt>, but as a
+ * cheap predicate on the parser's internal state, without materializing the
+ * partially parsed Ruby objects:
+ * parser << '{"a":1,'
+ * parser.parse # => false
+ * parser.partial_value? # => true
+ *
+ * A fully parsed document whose value hasn't been retrieved yet is not under
+ * construction: #value? returns true and #partial_value? returns false.
+ */
+static VALUE cResumableParser_partial_value_p(VALUE self)
+{
+ JSON_ResumableParser *parser = cResumableParser_get(self);
+
+ // Mirror of #value?: values on the stack while the document isn't DONE
+ // belong to a partially built document. A container whose first key or
+ // element hasn't been parsed yet has no frame nor value registered (the
+ // tokenizer rewinds to the container start on EOS), so that state is
+ // observable through the buffer (#eos?/#rest) instead, keeping this
+ // predicate consistent with #partial_value returning nil.
+ if (parser->value_stack.head > 0) {
+ json_frame *frame = json_frame_stack_peek(&parser->frames);
+ if (frame->phase != JSON_PHASE_DONE) {
+ return Qtrue;
+ }
+ }
+ return Qfalse;
+}
+
+/*
@@ -2780,0 +2875 @@
+ rb_define_method(cResumableParser, "partial_value?", cResumableParser_partial_value_p, 0);
@@ -2814,0 +2910 @@
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
@@ -2815,0 +2912 @@
+#endif
lib/json.rb
--- /tmp/d20260809-476-31vay6/json-2.20.0/lib/json.rb 2026-08-09 11:50:38.660698885 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/lib/json.rb 2026-08-09 11:50:38.801700618 +0000
@@ -124 +124,3 @@
-# defaults to +100+; specify +false+ to disable depth checking.
+# defaults to +100+;
+# You can set it to +false+ to disable depth checking entirely, but that is dangerous
+# when parsing untrusted input.
@@ -126 +128 @@
-# With the default, +false+:
+# With the default, +100+:
@@ -386,0 +389,9 @@
+# With +false+:
+# obj = []
+# obj[0] = obj
+# # Raises SystemStackError: stack level too deep
+# JSON.generate(obj, max_nesting: false)
+#
+# Setting +max_nesting+ to +false+ can lead to a stackoverflow and may leave the program
+# in an unrecoverable state. It is discouraged.
+#
@@ -411 +421,0 @@
-# defaults to the empty \String, <tt>''</tt>;
@@ -418,0 +429,5 @@
+# - Option +sort_keys+ (boolean or \Proc) controls whether and how the keys of a
+# hash are sorted when generating the output; defaults to <tt>false</tt>.
+# When +true+, keys are sorted lexicographically. When a \Proc, it receives
+# the entire \Hash and must return a \Hash with its pairs in the desired
+# order, allowing for arbitrary sort orders.
lib/json/common.rb
--- /tmp/d20260809-476-31vay6/json-2.20.0/lib/json/common.rb 2026-08-09 11:50:38.662698909 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/lib/json/common.rb 2026-08-09 11:50:38.802700630 +0000
@@ -157,0 +158,9 @@
+
+ # The default proc used when the +sort_keys+ generation option is +true+.
+ # It returns a new hash with the entries sorted by their keys.
+ sort_keys_proc = ->(hash) { hash.sort.to_h }
+ if defined?(::Ractor) && Ractor.respond_to?(:shareable_lambda)
+ sort_keys_proc = Ractor.shareable_lambda(&sort_keys_proc)
+ end
+ generator::State.default_sort_keys_proc = sort_keys_proc
+
lib/json/ext.rb
--- /tmp/d20260809-476-31vay6/json-2.20.0/lib/json/ext.rb 2026-08-09 11:50:38.662698909 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/lib/json/ext.rb 2026-08-09 11:50:38.802700630 +0000
@@ -43,0 +44,26 @@
+ if defined?(ResumableParser) # Not yet available on JRuby
+ class ResumableParser
+ # Returns whether the parser is entirely done: no unconsumed bytes in
+ # the buffer, no document under construction and no parsed value
+ # awaiting retrieval.
+ #
+ # The main use case is detecting a truncated stream once the input is
+ # exhausted:
+ #
+ # loop do
+ # begin
+ # parser << socket.readpartial(4096)
+ # rescue EOFError
+ # break
+ # end
+ # while parser.parse
+ # process(parser.value)
+ # end
+ # end
+ # warn "stream was truncated" unless parser.empty?
+ def empty?
+ eos? && !partial_value? && !value?
+ end
+ end
+ end
+
lib/json/ext/generator/state.rb
--- /tmp/d20260809-476-31vay6/json-2.20.0/lib/json/ext/generator/state.rb 2026-08-09 11:50:38.663698921 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/lib/json/ext/generator/state.rb 2026-08-09 11:50:38.803700642 +0000
@@ -56,0 +57 @@
+ sort_keys: sort_keys
lib/json/truffle_ruby/generator.rb
--- /tmp/d20260809-476-31vay6/json-2.20.0/lib/json/truffle_ruby/generator.rb 2026-08-09 11:50:38.664698934 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/lib/json/truffle_ruby/generator.rb 2026-08-09 11:50:38.803700642 +0000
@@ -113,0 +114,2 @@
+ singleton_class.attr_accessor :default_sort_keys_proc # :nodoc:
+
@@ -166,0 +169 @@
+ @sort_keys = false
@@ -201,0 +205,27 @@
+ # Controls key sorting in the generated JSON. If set to +true+, object
+ # keys are sorted by key lexicographically. If set to a Proc, it
+ # receives the entire Hash and must return a Hash with its pairs in the
+ # desired order.
+ attr_reader :sort_keys
+
+ def sort_keys=(value) # :nodoc:
+ type_error = false
+ @sort_keys = case value
+ when Proc
+ value
+ when true
+ State.default_sort_keys_proc
+ when nil, false
+ false
+ else
+ type_error = true
+ false
+ end
+
+ if type_error
+ raise TypeError, "The `sort_keys` argument must be a boolean or a Proc"
+ end
+
+ @sort_keys
+ end
+
@@ -287,0 +318 @@
+ self.sort_keys = opts[:sort_keys] if opts.key?(:sort_keys)
@@ -352 +383 @@
- !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj)
+ !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj) and !@sort_keys
@@ -354,0 +386,4 @@
+ if @sort_keys
+ obj = @sort_keys.call(obj)
+ end
+
lib/json/version.rb
--- /tmp/d20260809-476-31vay6/json-2.20.0/lib/json/version.rb 2026-08-09 11:50:38.664698934 +0000
+++ /tmp/d20260809-476-31vay6/json-2.21.2/lib/json/version.rb 2026-08-09 11:50:38.803700642 +0000
@@ -4 +4 @@
- VERSION = '2.20.0'
+ VERSION = '2.21.2' |
Contributor
gem compare --diff json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/CHANGES.md 2026-08-09 11:50:47.097577400 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/CHANGES.md 2026-08-09 11:50:47.107577552 +0000
@@ -4,0 +5,16 @@
+### 2026-07-31 (2.21.2)
+
+* Fix a use-after-free bug in `JSON::ResumableParser`. [GHSA-9hj4-r449-hfvc].
+
+### 2026-07-13 (2.21.1)
+
+* Fix a compilation issue on Window and Microsoft Visual C++.
+
+### 2026-07-12 (2.21.0)
+
+* `JSON.generate` now accept a `sort_keys` option, which takes either a boolean or a block.
+* Added `#empty?` and `#partial_value?` methods on `JSON::ResumableParser`.
+* Numerous correctness and performance fixes for `JSON::ResumableParser`.
+* Avoid triggering Ruby's `float out of range` warning when parsing out of range numbers.
+* Declare C types with Ruby 4.1 `RUBY_TYPED_THREAD_SAFE_FREE`.
+
README.md
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/README.md 2026-08-09 11:50:47.097577400 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/README.md 2026-08-09 11:50:47.108577567 +0000
@@ -88 +88 @@
-JSON.generate(Position.new(1, 2)) # => Position not allowed in JSON (JSON::GeneratorError)
+JSON.generate(Position.new(1, 2), strict: true) # => Position not allowed in JSON (JSON::GeneratorError)
@@ -120 +120 @@
-coder = JSON::Combining.new do |object, is_object_key|
+coder = JSON::Coder.new do |object, is_object_key|
@@ -123,2 +123,2 @@
- if !string.valid_encoding? || string.encoding != Encoding::UTF_8
- Base64.encode64(string)
+ if !object.valid_encoding? || object.encoding != Encoding::UTF_8
+ Base64.encode64(object)
@@ -126 +126 @@
- string
+ object
ext/json/ext/generator/generator.c
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/ext/json/ext/generator/generator.c 2026-08-09 11:50:47.098577415 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/ext/json/ext/generator/generator.c 2026-08-09 11:50:47.280580177 +0000
@@ -36,0 +37 @@
+ VALUE sort_keys;
@@ -39 +40 @@
-static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8;
+static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8, default_sort_keys_proc;
@@ -43 +44 @@
- sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json;
+ sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json, sym_sort_keys;
@@ -711,0 +713 @@
+ rb_gc_mark_movable(state->sort_keys);
@@ -722,0 +725 @@
+ state->sort_keys = rb_gc_location(state->sort_keys);
@@ -742 +745 @@
- .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -771,0 +775 @@
+ RB_OBJ_WRITTEN(vstate, Qundef, state->sort_keys);
@@ -1052,0 +1057,5 @@
+ if (RB_UNLIKELY(data->state->sort_keys)) {
+ obj = rb_proc_call_with_block(data->state->sort_keys, 1, &obj, Qnil);
+ Check_Type(obj, T_HASH);
+ }
+
@@ -1378,0 +1388 @@
+ RB_OBJ_WRITTEN(obj, Qundef, objState->sort_keys);
@@ -1724,0 +1735,49 @@
+static VALUE cState_set_default_sort_keys_proc(VALUE self, VALUE proc)
+{
+ if (!rb_obj_is_proc(proc)) {
+ rb_raise(rb_eTypeError, "sort_key_proc must be a Proc");
+ }
+ return default_sort_keys_proc = proc;
+}
+
+static VALUE normalize_sort_keys(VALUE value)
+{
+ if (rb_obj_is_proc(value)) {
+ return value;
+ } else if (value == Qtrue) {
+ return default_sort_keys_proc;
+ } else if (RTEST(value)) {
+ rb_raise(rb_eTypeError, "The `sort_keys` argument must be a boolean or a Proc");
+ } else {
+ return Qfalse;
+ }
+}
+
+/*
+ * call-seq: sort_keys
+ *
+ * Get the value of sort_keys.
+ */
+static VALUE cState_sort_keys_p(VALUE self)
+{
+ GET_STATE(self);
+ return state->sort_keys;
+}
+
+/*
+ * call-seq: sort_keys=(value)
+ *
+ * value is a boolean or a proc. If the value is the boolean true, object keys
+ * will be sorted lexicographically in ascending order.
+ *
+ * If the value is a proc, it receives the entire Hash and must return a Hash
+ * with its pairs in the desired order, allowing for arbitrary sorting.
+ */
+static VALUE cState_sort_keys_set(VALUE self, VALUE value)
+{
+ rb_check_frozen(self);
+ GET_STATE(self);
+ RB_OBJ_WRITE(self, &state->sort_keys, normalize_sort_keys(value));
+ return Qnil;
+}
+
@@ -1834,0 +1894,3 @@
+ else if (key == sym_sort_keys) {
+ state_write_value(data, &state->sort_keys, normalize_sort_keys(val));
+ }
@@ -1911,0 +1974,2 @@
+ rb_global_variable(&default_sort_keys_proc);
+
@@ -1920,0 +1985,2 @@
+ rb_define_singleton_method(cState, "default_sort_keys_proc=", cState_set_default_sort_keys_proc, 1);
+
@@ -1959,0 +2026,2 @@
+ rb_define_method(cState, "sort_keys", cState_sort_keys_p, 0);
+ rb_define_method(cState, "sort_keys=", cState_sort_keys_set, 1);
@@ -1988,0 +2057 @@
+ sym_sort_keys = ID2SYM(rb_intern("sort_keys"));
ext/json/ext/json.h
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/ext/json/ext/json.h 2026-08-09 11:50:47.098577415 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/ext/json/ext/json.h 2026-08-09 11:50:47.280580177 +0000
@@ -33,0 +34,4 @@
+#ifndef RUBY_TYPED_THREAD_SAFE_FREE
+#define RUBY_TYPED_THREAD_SAFE_FREE RUBY_TYPED_FREE_IMMEDIATELY
+#endif
+
ext/json/ext/parser/extconf.rb
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/ext/json/ext/parser/extconf.rb 2026-08-09 11:50:47.099577430 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/ext/json/ext/parser/extconf.rb 2026-08-09 11:50:47.280580177 +0000
@@ -18,16 +17,0 @@
-def have_builtin_func(name, check_expr, opt = "", &b)
- checking_for checking_message(name.funcall_style, nil, opt) do
- if try_compile(<<SRC, opt, &b)
-int foo;
-int main() { #{check_expr}; return 0; }
-SRC
- $defs.push(format("-DHAVE_BUILTIN_%s", name.tr_cpp))
- true
- else
- false
- end
- end
-end
-
-have_builtin_func("__builtin_clzll", "__builtin_clzll(0)")
-
@@ -47,0 +32,6 @@
+
+# Disable function outlining on clang to prevent some of the repeated instructions
+# in json_eat_whitespace being outlined into function calls.
+# Note: This verifies '-mno-outline' is accepted as a valid compiler flag
+# and will not pass it if unsupported.
+append_cflags("-mno-outline")
ext/json/ext/parser/parser.c
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/ext/json/ext/parser/parser.c 2026-08-09 11:50:47.099577430 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/ext/json/ext/parser/parser.c 2026-08-09 11:50:47.281580193 +0000
@@ -8 +8,4 @@
-static ID i_new, i_try_convert, i_uminus, i_encode, i_at_line, i_at_column;
+static ID i_new, i_try_convert, i_encode, i_at_line, i_at_column;
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
+static ID i_uminus;
+#endif
@@ -317 +320 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -514 +517 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
@@ -588,0 +592,2 @@
+ JSON_ASSERT(!state->parser);
+ JSON_ASSERT(state->cursor);
@@ -621,4 +626,8 @@
- long line, column;
- cursor_position(state, &line, &column);
-
- VALUE warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ VALUE warning;
+ if (state->parser) { // line and columns can't be accurate in resumable
+ warning = rb_utf8_str_new_cstr(message);
+ } else {
+ long line, column;
+ cursor_position(state, &line, &column);
+ warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ }
@@ -768 +777 @@
-json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config)
+json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config, const char *resume_pos)
@@ -774,0 +784,5 @@
+ // An incomplete comment suspends a resumable parse by rewinding the cursor
+ // and throwing. Callers that already consumed a token not yet committed to
+ // the frame stack pass resume_pos so the rewind re-reads that token too.
+ // Non-resumable error positions keep pointing at the comment either way.
+ const char *rewind_pos = (state->parser && resume_pos) ? resume_pos : start;
@@ -779,2 +793,9 @@
- state->cursor = memchr(state->cursor, '\n', state->end - state->cursor);
- if (!state->cursor) {
+ const char *newline = memchr(state->cursor, '\n', state->end - state->cursor);
+ if (!newline) {
+ // state->parser marks resumable mode, where the buffer end is only a
+ // chunk boundary: the terminating newline may still arrive, so leave
+ // the comment unterminated instead of consuming to end as a one-shot
+ // parse would.
+ if (state->parser) {
+ raise_eos_error_at("unterminated comment, expected end of line", state, rewind_pos);
+ }
@@ -783 +804 @@
- state->cursor++;
+ state->cursor = newline + 1;
@@ -793 +814 @@
- raise_eos_error_at("unterminated comment, expected closing '*/'", state, start);
+ raise_eos_error_at("unterminated comment, expected closing '*/'", state, rewind_pos);
@@ -805 +826 @@
- raise_parse_error_at("unexpected token %s", state, start, eos(state));
+ raise_parse_error_at("unexpected token %s", state, eos(state) ? rewind_pos : start, eos(state));
@@ -816 +837 @@
-json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+json_eat_whitespace_resume_at(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments, const char *resume_pos)
@@ -851 +872 @@
- json_eat_comments(state, config);
+ json_eat_comments(state, config, resume_pos);
@@ -859,0 +881,6 @@
+ALWAYS_INLINE(static) void
+json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+{
+ json_eat_whitespace_resume_at(state, config, include_comments, NULL);
+}
+
@@ -1132,0 +1160,7 @@
+ // If the value is so small that it definitely underflows to 0.0, return early
+ // to avoid triggering a "Float out of range" warning from rb_cstr_to_dbl.
+ // When mantissa_digits + exponent < -324, value < 10^(-324) < DBL_TRUE_MIN/2,
+ // so it rounds to 0 in IEEE 754 round-to-nearest.
+ if (RB_UNLIKELY(mantissa_digits + exponent < -324)) {
+ return rb_float_new(negative ? -0.0 : 0.0);
+ }
@@ -1432 +1466 @@
-static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start)
+static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start, bool resumable)
@@ -1488,0 +1523,10 @@
+ // A number touching the end of the buffer may still grow in a later chunk,
+ // so the caller will rewind and wait. Decoding it now would build a value
+ // -- for a long run of digits, an expensive bignum -- only to discard it,
+ // and repeating that on every resumed chunk is quadratic in the number's
+ // length. The digit scan above already advanced the cursor, which is all
+ // the caller needs to detect the incomplete number.
+ if (RB_UNLIKELY(resumable && eos(state))) {
+ return Qundef;
+ }
+
@@ -1565,0 +1610,7 @@
+ // A trailing comma lands us here expecting an element but finding the
+ // closing bracket; hand off to ARRAY_COMMA to close. An empty array
+ // closes inline at '[', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && frame->type == JSON_FRAME_ARRAY && peek(state) == ']') {
+ goto JSON_PHASE_ARRAY_COMMA;
+ }
+
@@ -1606 +1657 @@
- value = json_parse_number(state, config, true, value_start);
+ value = json_parse_number(state, config, true, value_start, resumable);
@@ -1629 +1680 @@
- value = json_parse_number(state, config, false, value_start);
+ value = json_parse_number(state, config, false, value_start, resumable);
@@ -1661 +1712,3 @@
- json_eat_whitespace(state, config, true);
+ // The '[' is consumed but its frame is only pushed below, so a
+ // comment suspending here must resume from the bracket.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1690 +1743,2 @@
- json_eat_whitespace(state, config, true);
+ // Same as '[': the frame is only pushed below.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1747,0 +1802,7 @@
+ // A trailing comma lands us here expecting a key but finding the closing
+ // brace; hand off to OBJECT_COMMA to close. An empty object closes inline
+ // at '{', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && peek(state) == '}') {
+ goto JSON_PHASE_OBJECT_COMMA;
+ }
+
@@ -1810,7 +1871,4 @@
- if (config->allow_trailing_comma) {
- json_eat_whitespace(state, config, true);
- if (peek(state) == ']') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_ARRAY_COMMA;
- }
- }
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_VALUE once the ']' is in the buffer.
@@ -1855,9 +1913,4 @@
- json_eat_whitespace(state, config, true);
-
- if (config->allow_trailing_comma) {
- if (peek(state) == '}') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_OBJECT_COMMA;
- }
- }
-
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_OBJECT_KEY once the '}' is in the buffer.
@@ -2183 +2236 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -2268 +2321 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -2521,0 +2575 @@
+ parser->state.start = parser->state.cursor = parser->state.end = 0;
@@ -2618 +2672,6 @@
- // Copy the value stack as we need to mutate it.
+ // Copy the value stack as we need to mutate it. The collapse loop folds each
+ // open container by popping its entries and pushing the single result, so a
+ // parent always reclaims its child's slot; head exceeds its live size by at
+ // most one, either for the missing-value placeholder pushed below or for the
+ // result of folding an empty innermost container. That one spare slot keeps
+ // rvalue_stack_push from growing (reallocating) this ALLOCV buffer.
@@ -2620,3 +2679,3 @@
- parser.value_stack.capa = (capa + missing_object_value);
- VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, capa + missing_object_value);
- MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, parser.value_stack.capa);
+ parser.value_stack.capa = capa + 1;
+ VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, parser.value_stack.capa);
+ MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, capa);
@@ -2715 +2774 @@
- * call-seq: value? -> true or false
+ * call-seq: eos? -> true or false
@@ -2725,0 +2785,35 @@
+ * call-seq: partial_value? -> true or false
+ *
+ * Returns whether a document is currently under construction: an unclosed
+ * container, a key awaiting its value, etc.
+ *
+ * It answers the same question as <tt>!partial_value.nil?</tt>, but as a
+ * cheap predicate on the parser's internal state, without materializing the
+ * partially parsed Ruby objects:
+ * parser << '{"a":1,'
+ * parser.parse # => false
+ * parser.partial_value? # => true
+ *
+ * A fully parsed document whose value hasn't been retrieved yet is not under
+ * construction: #value? returns true and #partial_value? returns false.
+ */
+static VALUE cResumableParser_partial_value_p(VALUE self)
+{
+ JSON_ResumableParser *parser = cResumableParser_get(self);
+
+ // Mirror of #value?: values on the stack while the document isn't DONE
+ // belong to a partially built document. A container whose first key or
+ // element hasn't been parsed yet has no frame nor value registered (the
+ // tokenizer rewinds to the container start on EOS), so that state is
+ // observable through the buffer (#eos?/#rest) instead, keeping this
+ // predicate consistent with #partial_value returning nil.
+ if (parser->value_stack.head > 0) {
+ json_frame *frame = json_frame_stack_peek(&parser->frames);
+ if (frame->phase != JSON_PHASE_DONE) {
+ return Qtrue;
+ }
+ }
+ return Qfalse;
+}
+
+/*
@@ -2780,0 +2875 @@
+ rb_define_method(cResumableParser, "partial_value?", cResumableParser_partial_value_p, 0);
@@ -2814,0 +2910 @@
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
@@ -2815,0 +2912 @@
+#endif
lib/json.rb
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/lib/json.rb 2026-08-09 11:50:47.102577476 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/lib/json.rb 2026-08-09 11:50:47.282580208 +0000
@@ -124 +124,3 @@
-# defaults to +100+; specify +false+ to disable depth checking.
+# defaults to +100+;
+# You can set it to +false+ to disable depth checking entirely, but that is dangerous
+# when parsing untrusted input.
@@ -126 +128 @@
-# With the default, +false+:
+# With the default, +100+:
@@ -386,0 +389,9 @@
+# With +false+:
+# obj = []
+# obj[0] = obj
+# # Raises SystemStackError: stack level too deep
+# JSON.generate(obj, max_nesting: false)
+#
+# Setting +max_nesting+ to +false+ can lead to a stackoverflow and may leave the program
+# in an unrecoverable state. It is discouraged.
+#
@@ -411 +421,0 @@
-# defaults to the empty \String, <tt>''</tt>;
@@ -418,0 +429,5 @@
+# - Option +sort_keys+ (boolean or \Proc) controls whether and how the keys of a
+# hash are sorted when generating the output; defaults to <tt>false</tt>.
+# When +true+, keys are sorted lexicographically. When a \Proc, it receives
+# the entire \Hash and must return a \Hash with its pairs in the desired
+# order, allowing for arbitrary sort orders.
lib/json/common.rb
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/lib/json/common.rb 2026-08-09 11:50:47.105577521 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/lib/json/common.rb 2026-08-09 11:50:47.283580223 +0000
@@ -157,0 +158,9 @@
+
+ # The default proc used when the +sort_keys+ generation option is +true+.
+ # It returns a new hash with the entries sorted by their keys.
+ sort_keys_proc = ->(hash) { hash.sort.to_h }
+ if defined?(::Ractor) && Ractor.respond_to?(:shareable_lambda)
+ sort_keys_proc = Ractor.shareable_lambda(&sort_keys_proc)
+ end
+ generator::State.default_sort_keys_proc = sort_keys_proc
+
lib/json/ext.rb
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/lib/json/ext.rb 2026-08-09 11:50:47.105577521 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/lib/json/ext.rb 2026-08-09 11:50:47.283580223 +0000
@@ -43,0 +44,26 @@
+ if defined?(ResumableParser) # Not yet available on JRuby
+ class ResumableParser
+ # Returns whether the parser is entirely done: no unconsumed bytes in
+ # the buffer, no document under construction and no parsed value
+ # awaiting retrieval.
+ #
+ # The main use case is detecting a truncated stream once the input is
+ # exhausted:
+ #
+ # loop do
+ # begin
+ # parser << socket.readpartial(4096)
+ # rescue EOFError
+ # break
+ # end
+ # while parser.parse
+ # process(parser.value)
+ # end
+ # end
+ # warn "stream was truncated" unless parser.empty?
+ def empty?
+ eos? && !partial_value? && !value?
+ end
+ end
+ end
+
lib/json/ext/generator/state.rb
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/lib/json/ext/generator/state.rb 2026-08-09 11:50:47.106577537 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/lib/json/ext/generator/state.rb 2026-08-09 11:50:47.283580223 +0000
@@ -56,0 +57 @@
+ sort_keys: sort_keys
lib/json/truffle_ruby/generator.rb
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/lib/json/truffle_ruby/generator.rb 2026-08-09 11:50:47.107577552 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/lib/json/truffle_ruby/generator.rb 2026-08-09 11:50:47.284580238 +0000
@@ -113,0 +114,2 @@
+ singleton_class.attr_accessor :default_sort_keys_proc # :nodoc:
+
@@ -166,0 +169 @@
+ @sort_keys = false
@@ -201,0 +205,27 @@
+ # Controls key sorting in the generated JSON. If set to +true+, object
+ # keys are sorted by key lexicographically. If set to a Proc, it
+ # receives the entire Hash and must return a Hash with its pairs in the
+ # desired order.
+ attr_reader :sort_keys
+
+ def sort_keys=(value) # :nodoc:
+ type_error = false
+ @sort_keys = case value
+ when Proc
+ value
+ when true
+ State.default_sort_keys_proc
+ when nil, false
+ false
+ else
+ type_error = true
+ false
+ end
+
+ if type_error
+ raise TypeError, "The `sort_keys` argument must be a boolean or a Proc"
+ end
+
+ @sort_keys
+ end
+
@@ -287,0 +318 @@
+ self.sort_keys = opts[:sort_keys] if opts.key?(:sort_keys)
@@ -352 +383 @@
- !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj)
+ !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj) and !@sort_keys
@@ -354,0 +386,4 @@
+ if @sort_keys
+ obj = @sort_keys.call(obj)
+ end
+
lib/json/version.rb
--- /tmp/d20260809-452-mpz8ns/json-2.20.0/lib/json/version.rb 2026-08-09 11:50:47.107577552 +0000
+++ /tmp/d20260809-452-mpz8ns/json-2.21.2/lib/json/version.rb 2026-08-09 11:50:47.284580238 +0000
@@ -4 +4 @@
- VERSION = '2.20.0'
+ VERSION = '2.21.2' |
Contributor
gem compare --diff json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/CHANGES.md 2026-08-09 11:51:00.880806084 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/CHANGES.md 2026-08-09 11:51:00.890806064 +0000
@@ -4,0 +5,16 @@
+### 2026-07-31 (2.21.2)
+
+* Fix a use-after-free bug in `JSON::ResumableParser`. [GHSA-9hj4-r449-hfvc].
+
+### 2026-07-13 (2.21.1)
+
+* Fix a compilation issue on Window and Microsoft Visual C++.
+
+### 2026-07-12 (2.21.0)
+
+* `JSON.generate` now accept a `sort_keys` option, which takes either a boolean or a block.
+* Added `#empty?` and `#partial_value?` methods on `JSON::ResumableParser`.
+* Numerous correctness and performance fixes for `JSON::ResumableParser`.
+* Avoid triggering Ruby's `float out of range` warning when parsing out of range numbers.
+* Declare C types with Ruby 4.1 `RUBY_TYPED_THREAD_SAFE_FREE`.
+
README.md
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/README.md 2026-08-09 11:51:00.880806084 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/README.md 2026-08-09 11:51:00.891806062 +0000
@@ -88 +88 @@
-JSON.generate(Position.new(1, 2)) # => Position not allowed in JSON (JSON::GeneratorError)
+JSON.generate(Position.new(1, 2), strict: true) # => Position not allowed in JSON (JSON::GeneratorError)
@@ -120 +120 @@
-coder = JSON::Combining.new do |object, is_object_key|
+coder = JSON::Coder.new do |object, is_object_key|
@@ -123,2 +123,2 @@
- if !string.valid_encoding? || string.encoding != Encoding::UTF_8
- Base64.encode64(string)
+ if !object.valid_encoding? || object.encoding != Encoding::UTF_8
+ Base64.encode64(object)
@@ -126 +126 @@
- string
+ object
ext/json/ext/generator/generator.c
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/ext/json/ext/generator/generator.c 2026-08-09 11:51:00.881806082 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/ext/json/ext/generator/generator.c 2026-08-09 11:51:00.892806060 +0000
@@ -36,0 +37 @@
+ VALUE sort_keys;
@@ -39 +40 @@
-static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8;
+static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8, default_sort_keys_proc;
@@ -43 +44 @@
- sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json;
+ sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json, sym_sort_keys;
@@ -711,0 +713 @@
+ rb_gc_mark_movable(state->sort_keys);
@@ -722,0 +725 @@
+ state->sort_keys = rb_gc_location(state->sort_keys);
@@ -742 +745 @@
- .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -771,0 +775 @@
+ RB_OBJ_WRITTEN(vstate, Qundef, state->sort_keys);
@@ -1052,0 +1057,5 @@
+ if (RB_UNLIKELY(data->state->sort_keys)) {
+ obj = rb_proc_call_with_block(data->state->sort_keys, 1, &obj, Qnil);
+ Check_Type(obj, T_HASH);
+ }
+
@@ -1378,0 +1388 @@
+ RB_OBJ_WRITTEN(obj, Qundef, objState->sort_keys);
@@ -1724,0 +1735,49 @@
+static VALUE cState_set_default_sort_keys_proc(VALUE self, VALUE proc)
+{
+ if (!rb_obj_is_proc(proc)) {
+ rb_raise(rb_eTypeError, "sort_key_proc must be a Proc");
+ }
+ return default_sort_keys_proc = proc;
+}
+
+static VALUE normalize_sort_keys(VALUE value)
+{
+ if (rb_obj_is_proc(value)) {
+ return value;
+ } else if (value == Qtrue) {
+ return default_sort_keys_proc;
+ } else if (RTEST(value)) {
+ rb_raise(rb_eTypeError, "The `sort_keys` argument must be a boolean or a Proc");
+ } else {
+ return Qfalse;
+ }
+}
+
+/*
+ * call-seq: sort_keys
+ *
+ * Get the value of sort_keys.
+ */
+static VALUE cState_sort_keys_p(VALUE self)
+{
+ GET_STATE(self);
+ return state->sort_keys;
+}
+
+/*
+ * call-seq: sort_keys=(value)
+ *
+ * value is a boolean or a proc. If the value is the boolean true, object keys
+ * will be sorted lexicographically in ascending order.
+ *
+ * If the value is a proc, it receives the entire Hash and must return a Hash
+ * with its pairs in the desired order, allowing for arbitrary sorting.
+ */
+static VALUE cState_sort_keys_set(VALUE self, VALUE value)
+{
+ rb_check_frozen(self);
+ GET_STATE(self);
+ RB_OBJ_WRITE(self, &state->sort_keys, normalize_sort_keys(value));
+ return Qnil;
+}
+
@@ -1834,0 +1894,3 @@
+ else if (key == sym_sort_keys) {
+ state_write_value(data, &state->sort_keys, normalize_sort_keys(val));
+ }
@@ -1911,0 +1974,2 @@
+ rb_global_variable(&default_sort_keys_proc);
+
@@ -1920,0 +1985,2 @@
+ rb_define_singleton_method(cState, "default_sort_keys_proc=", cState_set_default_sort_keys_proc, 1);
+
@@ -1959,0 +2026,2 @@
+ rb_define_method(cState, "sort_keys", cState_sort_keys_p, 0);
+ rb_define_method(cState, "sort_keys=", cState_sort_keys_set, 1);
@@ -1988,0 +2057 @@
+ sym_sort_keys = ID2SYM(rb_intern("sort_keys"));
ext/json/ext/json.h
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/ext/json/ext/json.h 2026-08-09 11:51:00.881806082 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/ext/json/ext/json.h 2026-08-09 11:51:00.892806060 +0000
@@ -33,0 +34,4 @@
+#ifndef RUBY_TYPED_THREAD_SAFE_FREE
+#define RUBY_TYPED_THREAD_SAFE_FREE RUBY_TYPED_FREE_IMMEDIATELY
+#endif
+
ext/json/ext/parser/extconf.rb
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/ext/json/ext/parser/extconf.rb 2026-08-09 11:51:00.882806080 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/ext/json/ext/parser/extconf.rb 2026-08-09 11:51:00.893806058 +0000
@@ -18,16 +17,0 @@
-def have_builtin_func(name, check_expr, opt = "", &b)
- checking_for checking_message(name.funcall_style, nil, opt) do
- if try_compile(<<SRC, opt, &b)
-int foo;
-int main() { #{check_expr}; return 0; }
-SRC
- $defs.push(format("-DHAVE_BUILTIN_%s", name.tr_cpp))
- true
- else
- false
- end
- end
-end
-
-have_builtin_func("__builtin_clzll", "__builtin_clzll(0)")
-
@@ -47,0 +32,6 @@
+
+# Disable function outlining on clang to prevent some of the repeated instructions
+# in json_eat_whitespace being outlined into function calls.
+# Note: This verifies '-mno-outline' is accepted as a valid compiler flag
+# and will not pass it if unsupported.
+append_cflags("-mno-outline")
ext/json/ext/parser/parser.c
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/ext/json/ext/parser/parser.c 2026-08-09 11:51:00.882806080 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/ext/json/ext/parser/parser.c 2026-08-09 11:51:01.028805790 +0000
@@ -8 +8,4 @@
-static ID i_new, i_try_convert, i_uminus, i_encode, i_at_line, i_at_column;
+static ID i_new, i_try_convert, i_encode, i_at_line, i_at_column;
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
+static ID i_uminus;
+#endif
@@ -317 +320 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -514 +517 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
@@ -588,0 +592,2 @@
+ JSON_ASSERT(!state->parser);
+ JSON_ASSERT(state->cursor);
@@ -621,4 +626,8 @@
- long line, column;
- cursor_position(state, &line, &column);
-
- VALUE warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ VALUE warning;
+ if (state->parser) { // line and columns can't be accurate in resumable
+ warning = rb_utf8_str_new_cstr(message);
+ } else {
+ long line, column;
+ cursor_position(state, &line, &column);
+ warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ }
@@ -768 +777 @@
-json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config)
+json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config, const char *resume_pos)
@@ -774,0 +784,5 @@
+ // An incomplete comment suspends a resumable parse by rewinding the cursor
+ // and throwing. Callers that already consumed a token not yet committed to
+ // the frame stack pass resume_pos so the rewind re-reads that token too.
+ // Non-resumable error positions keep pointing at the comment either way.
+ const char *rewind_pos = (state->parser && resume_pos) ? resume_pos : start;
@@ -779,2 +793,9 @@
- state->cursor = memchr(state->cursor, '\n', state->end - state->cursor);
- if (!state->cursor) {
+ const char *newline = memchr(state->cursor, '\n', state->end - state->cursor);
+ if (!newline) {
+ // state->parser marks resumable mode, where the buffer end is only a
+ // chunk boundary: the terminating newline may still arrive, so leave
+ // the comment unterminated instead of consuming to end as a one-shot
+ // parse would.
+ if (state->parser) {
+ raise_eos_error_at("unterminated comment, expected end of line", state, rewind_pos);
+ }
@@ -783 +804 @@
- state->cursor++;
+ state->cursor = newline + 1;
@@ -793 +814 @@
- raise_eos_error_at("unterminated comment, expected closing '*/'", state, start);
+ raise_eos_error_at("unterminated comment, expected closing '*/'", state, rewind_pos);
@@ -805 +826 @@
- raise_parse_error_at("unexpected token %s", state, start, eos(state));
+ raise_parse_error_at("unexpected token %s", state, eos(state) ? rewind_pos : start, eos(state));
@@ -816 +837 @@
-json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+json_eat_whitespace_resume_at(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments, const char *resume_pos)
@@ -851 +872 @@
- json_eat_comments(state, config);
+ json_eat_comments(state, config, resume_pos);
@@ -859,0 +881,6 @@
+ALWAYS_INLINE(static) void
+json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+{
+ json_eat_whitespace_resume_at(state, config, include_comments, NULL);
+}
+
@@ -1132,0 +1160,7 @@
+ // If the value is so small that it definitely underflows to 0.0, return early
+ // to avoid triggering a "Float out of range" warning from rb_cstr_to_dbl.
+ // When mantissa_digits + exponent < -324, value < 10^(-324) < DBL_TRUE_MIN/2,
+ // so it rounds to 0 in IEEE 754 round-to-nearest.
+ if (RB_UNLIKELY(mantissa_digits + exponent < -324)) {
+ return rb_float_new(negative ? -0.0 : 0.0);
+ }
@@ -1432 +1466 @@
-static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start)
+static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start, bool resumable)
@@ -1488,0 +1523,10 @@
+ // A number touching the end of the buffer may still grow in a later chunk,
+ // so the caller will rewind and wait. Decoding it now would build a value
+ // -- for a long run of digits, an expensive bignum -- only to discard it,
+ // and repeating that on every resumed chunk is quadratic in the number's
+ // length. The digit scan above already advanced the cursor, which is all
+ // the caller needs to detect the incomplete number.
+ if (RB_UNLIKELY(resumable && eos(state))) {
+ return Qundef;
+ }
+
@@ -1565,0 +1610,7 @@
+ // A trailing comma lands us here expecting an element but finding the
+ // closing bracket; hand off to ARRAY_COMMA to close. An empty array
+ // closes inline at '[', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && frame->type == JSON_FRAME_ARRAY && peek(state) == ']') {
+ goto JSON_PHASE_ARRAY_COMMA;
+ }
+
@@ -1606 +1657 @@
- value = json_parse_number(state, config, true, value_start);
+ value = json_parse_number(state, config, true, value_start, resumable);
@@ -1629 +1680 @@
- value = json_parse_number(state, config, false, value_start);
+ value = json_parse_number(state, config, false, value_start, resumable);
@@ -1661 +1712,3 @@
- json_eat_whitespace(state, config, true);
+ // The '[' is consumed but its frame is only pushed below, so a
+ // comment suspending here must resume from the bracket.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1690 +1743,2 @@
- json_eat_whitespace(state, config, true);
+ // Same as '[': the frame is only pushed below.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1747,0 +1802,7 @@
+ // A trailing comma lands us here expecting a key but finding the closing
+ // brace; hand off to OBJECT_COMMA to close. An empty object closes inline
+ // at '{', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && peek(state) == '}') {
+ goto JSON_PHASE_OBJECT_COMMA;
+ }
+
@@ -1810,7 +1871,4 @@
- if (config->allow_trailing_comma) {
- json_eat_whitespace(state, config, true);
- if (peek(state) == ']') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_ARRAY_COMMA;
- }
- }
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_VALUE once the ']' is in the buffer.
@@ -1855,9 +1913,4 @@
- json_eat_whitespace(state, config, true);
-
- if (config->allow_trailing_comma) {
- if (peek(state) == '}') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_OBJECT_COMMA;
- }
- }
-
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_OBJECT_KEY once the '}' is in the buffer.
@@ -2183 +2236 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -2268 +2321 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -2521,0 +2575 @@
+ parser->state.start = parser->state.cursor = parser->state.end = 0;
@@ -2618 +2672,6 @@
- // Copy the value stack as we need to mutate it.
+ // Copy the value stack as we need to mutate it. The collapse loop folds each
+ // open container by popping its entries and pushing the single result, so a
+ // parent always reclaims its child's slot; head exceeds its live size by at
+ // most one, either for the missing-value placeholder pushed below or for the
+ // result of folding an empty innermost container. That one spare slot keeps
+ // rvalue_stack_push from growing (reallocating) this ALLOCV buffer.
@@ -2620,3 +2679,3 @@
- parser.value_stack.capa = (capa + missing_object_value);
- VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, capa + missing_object_value);
- MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, parser.value_stack.capa);
+ parser.value_stack.capa = capa + 1;
+ VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, parser.value_stack.capa);
+ MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, capa);
@@ -2715 +2774 @@
- * call-seq: value? -> true or false
+ * call-seq: eos? -> true or false
@@ -2725,0 +2785,35 @@
+ * call-seq: partial_value? -> true or false
+ *
+ * Returns whether a document is currently under construction: an unclosed
+ * container, a key awaiting its value, etc.
+ *
+ * It answers the same question as <tt>!partial_value.nil?</tt>, but as a
+ * cheap predicate on the parser's internal state, without materializing the
+ * partially parsed Ruby objects:
+ * parser << '{"a":1,'
+ * parser.parse # => false
+ * parser.partial_value? # => true
+ *
+ * A fully parsed document whose value hasn't been retrieved yet is not under
+ * construction: #value? returns true and #partial_value? returns false.
+ */
+static VALUE cResumableParser_partial_value_p(VALUE self)
+{
+ JSON_ResumableParser *parser = cResumableParser_get(self);
+
+ // Mirror of #value?: values on the stack while the document isn't DONE
+ // belong to a partially built document. A container whose first key or
+ // element hasn't been parsed yet has no frame nor value registered (the
+ // tokenizer rewinds to the container start on EOS), so that state is
+ // observable through the buffer (#eos?/#rest) instead, keeping this
+ // predicate consistent with #partial_value returning nil.
+ if (parser->value_stack.head > 0) {
+ json_frame *frame = json_frame_stack_peek(&parser->frames);
+ if (frame->phase != JSON_PHASE_DONE) {
+ return Qtrue;
+ }
+ }
+ return Qfalse;
+}
+
+/*
@@ -2780,0 +2875 @@
+ rb_define_method(cResumableParser, "partial_value?", cResumableParser_partial_value_p, 0);
@@ -2814,0 +2910 @@
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
@@ -2815,0 +2912 @@
+#endif
lib/json.rb
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/lib/json.rb 2026-08-09 11:51:00.885806074 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/lib/json.rb 2026-08-09 11:51:01.029805787 +0000
@@ -124 +124,3 @@
-# defaults to +100+; specify +false+ to disable depth checking.
+# defaults to +100+;
+# You can set it to +false+ to disable depth checking entirely, but that is dangerous
+# when parsing untrusted input.
@@ -126 +128 @@
-# With the default, +false+:
+# With the default, +100+:
@@ -386,0 +389,9 @@
+# With +false+:
+# obj = []
+# obj[0] = obj
+# # Raises SystemStackError: stack level too deep
+# JSON.generate(obj, max_nesting: false)
+#
+# Setting +max_nesting+ to +false+ can lead to a stackoverflow and may leave the program
+# in an unrecoverable state. It is discouraged.
+#
@@ -411 +421,0 @@
-# defaults to the empty \String, <tt>''</tt>;
@@ -418,0 +429,5 @@
+# - Option +sort_keys+ (boolean or \Proc) controls whether and how the keys of a
+# hash are sorted when generating the output; defaults to <tt>false</tt>.
+# When +true+, keys are sorted lexicographically. When a \Proc, it receives
+# the entire \Hash and must return a \Hash with its pairs in the desired
+# order, allowing for arbitrary sort orders.
lib/json/common.rb
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/lib/json/common.rb 2026-08-09 11:51:00.887806070 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/lib/json/common.rb 2026-08-09 11:51:01.031805784 +0000
@@ -157,0 +158,9 @@
+
+ # The default proc used when the +sort_keys+ generation option is +true+.
+ # It returns a new hash with the entries sorted by their keys.
+ sort_keys_proc = ->(hash) { hash.sort.to_h }
+ if defined?(::Ractor) && Ractor.respond_to?(:shareable_lambda)
+ sort_keys_proc = Ractor.shareable_lambda(&sort_keys_proc)
+ end
+ generator::State.default_sort_keys_proc = sort_keys_proc
+
lib/json/ext.rb
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/lib/json/ext.rb 2026-08-09 11:51:00.888806068 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/lib/json/ext.rb 2026-08-09 11:51:01.031805784 +0000
@@ -43,0 +44,26 @@
+ if defined?(ResumableParser) # Not yet available on JRuby
+ class ResumableParser
+ # Returns whether the parser is entirely done: no unconsumed bytes in
+ # the buffer, no document under construction and no parsed value
+ # awaiting retrieval.
+ #
+ # The main use case is detecting a truncated stream once the input is
+ # exhausted:
+ #
+ # loop do
+ # begin
+ # parser << socket.readpartial(4096)
+ # rescue EOFError
+ # break
+ # end
+ # while parser.parse
+ # process(parser.value)
+ # end
+ # end
+ # warn "stream was truncated" unless parser.empty?
+ def empty?
+ eos? && !partial_value? && !value?
+ end
+ end
+ end
+
lib/json/ext/generator/state.rb
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/lib/json/ext/generator/state.rb 2026-08-09 11:51:00.889806066 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/lib/json/ext/generator/state.rb 2026-08-09 11:51:01.032805782 +0000
@@ -56,0 +57 @@
+ sort_keys: sort_keys
lib/json/truffle_ruby/generator.rb
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/lib/json/truffle_ruby/generator.rb 2026-08-09 11:51:00.889806066 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/lib/json/truffle_ruby/generator.rb 2026-08-09 11:51:01.032805782 +0000
@@ -113,0 +114,2 @@
+ singleton_class.attr_accessor :default_sort_keys_proc # :nodoc:
+
@@ -166,0 +169 @@
+ @sort_keys = false
@@ -201,0 +205,27 @@
+ # Controls key sorting in the generated JSON. If set to +true+, object
+ # keys are sorted by key lexicographically. If set to a Proc, it
+ # receives the entire Hash and must return a Hash with its pairs in the
+ # desired order.
+ attr_reader :sort_keys
+
+ def sort_keys=(value) # :nodoc:
+ type_error = false
+ @sort_keys = case value
+ when Proc
+ value
+ when true
+ State.default_sort_keys_proc
+ when nil, false
+ false
+ else
+ type_error = true
+ false
+ end
+
+ if type_error
+ raise TypeError, "The `sort_keys` argument must be a boolean or a Proc"
+ end
+
+ @sort_keys
+ end
+
@@ -287,0 +318 @@
+ self.sort_keys = opts[:sort_keys] if opts.key?(:sort_keys)
@@ -352 +383 @@
- !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj)
+ !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj) and !@sort_keys
@@ -354,0 +386,4 @@
+ if @sort_keys
+ obj = @sort_keys.call(obj)
+ end
+
lib/json/version.rb
--- /tmp/d20260809-472-iwt9s1/json-2.20.0/lib/json/version.rb 2026-08-09 11:51:00.889806066 +0000
+++ /tmp/d20260809-472-iwt9s1/json-2.21.2/lib/json/version.rb 2026-08-09 11:51:01.032805782 +0000
@@ -4 +4 @@
- VERSION = '2.20.0'
+ VERSION = '2.21.2' |
Contributor
gem compare json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT require_paths:
2.20.0: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.20.0", "lib"]
2.21.2: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.21.2", "lib"]
DIFFERENT rubygems_version:
2.20.0: 4.0.12
2.21.2: 4.0.16
DIFFERENT version:
2.20.0: 2.20.0
2.21.2: 2.21.2
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md +16/-0
README.md +5/-5
ext/json/ext/generator/generator.c +72/-3
ext/json/ext/json.h +4/-0
ext/json/ext/parser/extconf.rb +6/-16
ext/json/ext/parser/parser.c +140/-43
lib/json.rb +18/-3
lib/json/common.rb +9/-0
lib/json/ext.rb +26/-0
lib/json/ext/generator/state.rb +1/-0
lib/json/truffle_ruby/generator.rb +36/-1
lib/json/version.rb +1/-1
DIFFERENT extra_rdoc_files:
2.20.0->2.21.2:
* Changed:
README.md +5/-5 |
1 similar comment
Contributor
gem compare json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT require_paths:
2.20.0: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.20.0", "lib"]
2.21.2: ["/opt/hostedtoolcache/Ruby/4.0.6/x64/lib/ruby/gems/4.0.0/extensions/x86_64-linux/4.0.0/json-2.21.2", "lib"]
DIFFERENT rubygems_version:
2.20.0: 4.0.12
2.21.2: 4.0.16
DIFFERENT version:
2.20.0: 2.20.0
2.21.2: 2.21.2
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md +16/-0
README.md +5/-5
ext/json/ext/generator/generator.c +72/-3
ext/json/ext/json.h +4/-0
ext/json/ext/parser/extconf.rb +6/-16
ext/json/ext/parser/parser.c +140/-43
lib/json.rb +18/-3
lib/json/common.rb +9/-0
lib/json/ext.rb +26/-0
lib/json/ext/generator/state.rb +1/-0
lib/json/truffle_ruby/generator.rb +36/-1
lib/json/version.rb +1/-1
DIFFERENT extra_rdoc_files:
2.20.0->2.21.2:
* Changed:
README.md +5/-5 |
Contributor
gem compare --diff json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/CHANGES.md 2026-08-09 11:51:19.721162189 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/CHANGES.md 2026-08-09 11:51:19.731162262 +0000
@@ -4,0 +5,16 @@
+### 2026-07-31 (2.21.2)
+
+* Fix a use-after-free bug in `JSON::ResumableParser`. [GHSA-9hj4-r449-hfvc].
+
+### 2026-07-13 (2.21.1)
+
+* Fix a compilation issue on Window and Microsoft Visual C++.
+
+### 2026-07-12 (2.21.0)
+
+* `JSON.generate` now accept a `sort_keys` option, which takes either a boolean or a block.
+* Added `#empty?` and `#partial_value?` methods on `JSON::ResumableParser`.
+* Numerous correctness and performance fixes for `JSON::ResumableParser`.
+* Avoid triggering Ruby's `float out of range` warning when parsing out of range numbers.
+* Declare C types with Ruby 4.1 `RUBY_TYPED_THREAD_SAFE_FREE`.
+
README.md
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/README.md 2026-08-09 11:51:19.722162196 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/README.md 2026-08-09 11:51:19.731162262 +0000
@@ -88 +88 @@
-JSON.generate(Position.new(1, 2)) # => Position not allowed in JSON (JSON::GeneratorError)
+JSON.generate(Position.new(1, 2), strict: true) # => Position not allowed in JSON (JSON::GeneratorError)
@@ -120 +120 @@
-coder = JSON::Combining.new do |object, is_object_key|
+coder = JSON::Coder.new do |object, is_object_key|
@@ -123,2 +123,2 @@
- if !string.valid_encoding? || string.encoding != Encoding::UTF_8
- Base64.encode64(string)
+ if !object.valid_encoding? || object.encoding != Encoding::UTF_8
+ Base64.encode64(object)
@@ -126 +126 @@
- string
+ object
ext/json/ext/generator/generator.c
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/ext/json/ext/generator/generator.c 2026-08-09 11:51:19.722162196 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/ext/json/ext/generator/generator.c 2026-08-09 11:51:19.733162276 +0000
@@ -36,0 +37 @@
+ VALUE sort_keys;
@@ -39 +40 @@
-static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8;
+static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8, default_sort_keys_proc;
@@ -43 +44 @@
- sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json;
+ sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json, sym_sort_keys;
@@ -711,0 +713 @@
+ rb_gc_mark_movable(state->sort_keys);
@@ -722,0 +725 @@
+ state->sort_keys = rb_gc_location(state->sort_keys);
@@ -742 +745 @@
- .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -771,0 +775 @@
+ RB_OBJ_WRITTEN(vstate, Qundef, state->sort_keys);
@@ -1052,0 +1057,5 @@
+ if (RB_UNLIKELY(data->state->sort_keys)) {
+ obj = rb_proc_call_with_block(data->state->sort_keys, 1, &obj, Qnil);
+ Check_Type(obj, T_HASH);
+ }
+
@@ -1378,0 +1388 @@
+ RB_OBJ_WRITTEN(obj, Qundef, objState->sort_keys);
@@ -1724,0 +1735,49 @@
+static VALUE cState_set_default_sort_keys_proc(VALUE self, VALUE proc)
+{
+ if (!rb_obj_is_proc(proc)) {
+ rb_raise(rb_eTypeError, "sort_key_proc must be a Proc");
+ }
+ return default_sort_keys_proc = proc;
+}
+
+static VALUE normalize_sort_keys(VALUE value)
+{
+ if (rb_obj_is_proc(value)) {
+ return value;
+ } else if (value == Qtrue) {
+ return default_sort_keys_proc;
+ } else if (RTEST(value)) {
+ rb_raise(rb_eTypeError, "The `sort_keys` argument must be a boolean or a Proc");
+ } else {
+ return Qfalse;
+ }
+}
+
+/*
+ * call-seq: sort_keys
+ *
+ * Get the value of sort_keys.
+ */
+static VALUE cState_sort_keys_p(VALUE self)
+{
+ GET_STATE(self);
+ return state->sort_keys;
+}
+
+/*
+ * call-seq: sort_keys=(value)
+ *
+ * value is a boolean or a proc. If the value is the boolean true, object keys
+ * will be sorted lexicographically in ascending order.
+ *
+ * If the value is a proc, it receives the entire Hash and must return a Hash
+ * with its pairs in the desired order, allowing for arbitrary sorting.
+ */
+static VALUE cState_sort_keys_set(VALUE self, VALUE value)
+{
+ rb_check_frozen(self);
+ GET_STATE(self);
+ RB_OBJ_WRITE(self, &state->sort_keys, normalize_sort_keys(value));
+ return Qnil;
+}
+
@@ -1834,0 +1894,3 @@
+ else if (key == sym_sort_keys) {
+ state_write_value(data, &state->sort_keys, normalize_sort_keys(val));
+ }
@@ -1911,0 +1974,2 @@
+ rb_global_variable(&default_sort_keys_proc);
+
@@ -1920,0 +1985,2 @@
+ rb_define_singleton_method(cState, "default_sort_keys_proc=", cState_set_default_sort_keys_proc, 1);
+
@@ -1959,0 +2026,2 @@
+ rb_define_method(cState, "sort_keys", cState_sort_keys_p, 0);
+ rb_define_method(cState, "sort_keys=", cState_sort_keys_set, 1);
@@ -1988,0 +2057 @@
+ sym_sort_keys = ID2SYM(rb_intern("sort_keys"));
ext/json/ext/json.h
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/ext/json/ext/json.h 2026-08-09 11:51:19.723162203 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/ext/json/ext/json.h 2026-08-09 11:51:19.733162276 +0000
@@ -33,0 +34,4 @@
+#ifndef RUBY_TYPED_THREAD_SAFE_FREE
+#define RUBY_TYPED_THREAD_SAFE_FREE RUBY_TYPED_FREE_IMMEDIATELY
+#endif
+
ext/json/ext/parser/extconf.rb
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/ext/json/ext/parser/extconf.rb 2026-08-09 11:51:19.723162203 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/ext/json/ext/parser/extconf.rb 2026-08-09 11:51:19.876163322 +0000
@@ -18,16 +17,0 @@
-def have_builtin_func(name, check_expr, opt = "", &b)
- checking_for checking_message(name.funcall_style, nil, opt) do
- if try_compile(<<SRC, opt, &b)
-int foo;
-int main() { #{check_expr}; return 0; }
-SRC
- $defs.push(format("-DHAVE_BUILTIN_%s", name.tr_cpp))
- true
- else
- false
- end
- end
-end
-
-have_builtin_func("__builtin_clzll", "__builtin_clzll(0)")
-
@@ -47,0 +32,6 @@
+
+# Disable function outlining on clang to prevent some of the repeated instructions
+# in json_eat_whitespace being outlined into function calls.
+# Note: This verifies '-mno-outline' is accepted as a valid compiler flag
+# and will not pass it if unsupported.
+append_cflags("-mno-outline")
ext/json/ext/parser/parser.c
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/ext/json/ext/parser/parser.c 2026-08-09 11:51:19.724162211 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/ext/json/ext/parser/parser.c 2026-08-09 11:51:19.877163329 +0000
@@ -8 +8,4 @@
-static ID i_new, i_try_convert, i_uminus, i_encode, i_at_line, i_at_column;
+static ID i_new, i_try_convert, i_encode, i_at_line, i_at_column;
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
+static ID i_uminus;
+#endif
@@ -317 +320 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -514 +517 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
@@ -588,0 +592,2 @@
+ JSON_ASSERT(!state->parser);
+ JSON_ASSERT(state->cursor);
@@ -621,4 +626,8 @@
- long line, column;
- cursor_position(state, &line, &column);
-
- VALUE warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ VALUE warning;
+ if (state->parser) { // line and columns can't be accurate in resumable
+ warning = rb_utf8_str_new_cstr(message);
+ } else {
+ long line, column;
+ cursor_position(state, &line, &column);
+ warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ }
@@ -768 +777 @@
-json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config)
+json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config, const char *resume_pos)
@@ -774,0 +784,5 @@
+ // An incomplete comment suspends a resumable parse by rewinding the cursor
+ // and throwing. Callers that already consumed a token not yet committed to
+ // the frame stack pass resume_pos so the rewind re-reads that token too.
+ // Non-resumable error positions keep pointing at the comment either way.
+ const char *rewind_pos = (state->parser && resume_pos) ? resume_pos : start;
@@ -779,2 +793,9 @@
- state->cursor = memchr(state->cursor, '\n', state->end - state->cursor);
- if (!state->cursor) {
+ const char *newline = memchr(state->cursor, '\n', state->end - state->cursor);
+ if (!newline) {
+ // state->parser marks resumable mode, where the buffer end is only a
+ // chunk boundary: the terminating newline may still arrive, so leave
+ // the comment unterminated instead of consuming to end as a one-shot
+ // parse would.
+ if (state->parser) {
+ raise_eos_error_at("unterminated comment, expected end of line", state, rewind_pos);
+ }
@@ -783 +804 @@
- state->cursor++;
+ state->cursor = newline + 1;
@@ -793 +814 @@
- raise_eos_error_at("unterminated comment, expected closing '*/'", state, start);
+ raise_eos_error_at("unterminated comment, expected closing '*/'", state, rewind_pos);
@@ -805 +826 @@
- raise_parse_error_at("unexpected token %s", state, start, eos(state));
+ raise_parse_error_at("unexpected token %s", state, eos(state) ? rewind_pos : start, eos(state));
@@ -816 +837 @@
-json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+json_eat_whitespace_resume_at(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments, const char *resume_pos)
@@ -851 +872 @@
- json_eat_comments(state, config);
+ json_eat_comments(state, config, resume_pos);
@@ -859,0 +881,6 @@
+ALWAYS_INLINE(static) void
+json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+{
+ json_eat_whitespace_resume_at(state, config, include_comments, NULL);
+}
+
@@ -1132,0 +1160,7 @@
+ // If the value is so small that it definitely underflows to 0.0, return early
+ // to avoid triggering a "Float out of range" warning from rb_cstr_to_dbl.
+ // When mantissa_digits + exponent < -324, value < 10^(-324) < DBL_TRUE_MIN/2,
+ // so it rounds to 0 in IEEE 754 round-to-nearest.
+ if (RB_UNLIKELY(mantissa_digits + exponent < -324)) {
+ return rb_float_new(negative ? -0.0 : 0.0);
+ }
@@ -1432 +1466 @@
-static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start)
+static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start, bool resumable)
@@ -1488,0 +1523,10 @@
+ // A number touching the end of the buffer may still grow in a later chunk,
+ // so the caller will rewind and wait. Decoding it now would build a value
+ // -- for a long run of digits, an expensive bignum -- only to discard it,
+ // and repeating that on every resumed chunk is quadratic in the number's
+ // length. The digit scan above already advanced the cursor, which is all
+ // the caller needs to detect the incomplete number.
+ if (RB_UNLIKELY(resumable && eos(state))) {
+ return Qundef;
+ }
+
@@ -1565,0 +1610,7 @@
+ // A trailing comma lands us here expecting an element but finding the
+ // closing bracket; hand off to ARRAY_COMMA to close. An empty array
+ // closes inline at '[', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && frame->type == JSON_FRAME_ARRAY && peek(state) == ']') {
+ goto JSON_PHASE_ARRAY_COMMA;
+ }
+
@@ -1606 +1657 @@
- value = json_parse_number(state, config, true, value_start);
+ value = json_parse_number(state, config, true, value_start, resumable);
@@ -1629 +1680 @@
- value = json_parse_number(state, config, false, value_start);
+ value = json_parse_number(state, config, false, value_start, resumable);
@@ -1661 +1712,3 @@
- json_eat_whitespace(state, config, true);
+ // The '[' is consumed but its frame is only pushed below, so a
+ // comment suspending here must resume from the bracket.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1690 +1743,2 @@
- json_eat_whitespace(state, config, true);
+ // Same as '[': the frame is only pushed below.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1747,0 +1802,7 @@
+ // A trailing comma lands us here expecting a key but finding the closing
+ // brace; hand off to OBJECT_COMMA to close. An empty object closes inline
+ // at '{', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && peek(state) == '}') {
+ goto JSON_PHASE_OBJECT_COMMA;
+ }
+
@@ -1810,7 +1871,4 @@
- if (config->allow_trailing_comma) {
- json_eat_whitespace(state, config, true);
- if (peek(state) == ']') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_ARRAY_COMMA;
- }
- }
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_VALUE once the ']' is in the buffer.
@@ -1855,9 +1913,4 @@
- json_eat_whitespace(state, config, true);
-
- if (config->allow_trailing_comma) {
- if (peek(state) == '}') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_OBJECT_COMMA;
- }
- }
-
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_OBJECT_KEY once the '}' is in the buffer.
@@ -2183 +2236 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -2268 +2321 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -2521,0 +2575 @@
+ parser->state.start = parser->state.cursor = parser->state.end = 0;
@@ -2618 +2672,6 @@
- // Copy the value stack as we need to mutate it.
+ // Copy the value stack as we need to mutate it. The collapse loop folds each
+ // open container by popping its entries and pushing the single result, so a
+ // parent always reclaims its child's slot; head exceeds its live size by at
+ // most one, either for the missing-value placeholder pushed below or for the
+ // result of folding an empty innermost container. That one spare slot keeps
+ // rvalue_stack_push from growing (reallocating) this ALLOCV buffer.
@@ -2620,3 +2679,3 @@
- parser.value_stack.capa = (capa + missing_object_value);
- VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, capa + missing_object_value);
- MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, parser.value_stack.capa);
+ parser.value_stack.capa = capa + 1;
+ VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, parser.value_stack.capa);
+ MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, capa);
@@ -2715 +2774 @@
- * call-seq: value? -> true or false
+ * call-seq: eos? -> true or false
@@ -2725,0 +2785,35 @@
+ * call-seq: partial_value? -> true or false
+ *
+ * Returns whether a document is currently under construction: an unclosed
+ * container, a key awaiting its value, etc.
+ *
+ * It answers the same question as <tt>!partial_value.nil?</tt>, but as a
+ * cheap predicate on the parser's internal state, without materializing the
+ * partially parsed Ruby objects:
+ * parser << '{"a":1,'
+ * parser.parse # => false
+ * parser.partial_value? # => true
+ *
+ * A fully parsed document whose value hasn't been retrieved yet is not under
+ * construction: #value? returns true and #partial_value? returns false.
+ */
+static VALUE cResumableParser_partial_value_p(VALUE self)
+{
+ JSON_ResumableParser *parser = cResumableParser_get(self);
+
+ // Mirror of #value?: values on the stack while the document isn't DONE
+ // belong to a partially built document. A container whose first key or
+ // element hasn't been parsed yet has no frame nor value registered (the
+ // tokenizer rewinds to the container start on EOS), so that state is
+ // observable through the buffer (#eos?/#rest) instead, keeping this
+ // predicate consistent with #partial_value returning nil.
+ if (parser->value_stack.head > 0) {
+ json_frame *frame = json_frame_stack_peek(&parser->frames);
+ if (frame->phase != JSON_PHASE_DONE) {
+ return Qtrue;
+ }
+ }
+ return Qfalse;
+}
+
+/*
@@ -2780,0 +2875 @@
+ rb_define_method(cResumableParser, "partial_value?", cResumableParser_partial_value_p, 0);
@@ -2814,0 +2910 @@
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
@@ -2815,0 +2912 @@
+#endif
lib/json.rb
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/lib/json.rb 2026-08-09 11:51:19.726162225 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/lib/json.rb 2026-08-09 11:51:19.878163336 +0000
@@ -124 +124,3 @@
-# defaults to +100+; specify +false+ to disable depth checking.
+# defaults to +100+;
+# You can set it to +false+ to disable depth checking entirely, but that is dangerous
+# when parsing untrusted input.
@@ -126 +128 @@
-# With the default, +false+:
+# With the default, +100+:
@@ -386,0 +389,9 @@
+# With +false+:
+# obj = []
+# obj[0] = obj
+# # Raises SystemStackError: stack level too deep
+# JSON.generate(obj, max_nesting: false)
+#
+# Setting +max_nesting+ to +false+ can lead to a stackoverflow and may leave the program
+# in an unrecoverable state. It is discouraged.
+#
@@ -411 +421,0 @@
-# defaults to the empty \String, <tt>''</tt>;
@@ -418,0 +429,5 @@
+# - Option +sort_keys+ (boolean or \Proc) controls whether and how the keys of a
+# hash are sorted when generating the output; defaults to <tt>false</tt>.
+# When +true+, keys are sorted lexicographically. When a \Proc, it receives
+# the entire \Hash and must return a \Hash with its pairs in the desired
+# order, allowing for arbitrary sort orders.
lib/json/common.rb
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/lib/json/common.rb 2026-08-09 11:51:19.729162247 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/lib/json/common.rb 2026-08-09 11:51:19.880163351 +0000
@@ -157,0 +158,9 @@
+
+ # The default proc used when the +sort_keys+ generation option is +true+.
+ # It returns a new hash with the entries sorted by their keys.
+ sort_keys_proc = ->(hash) { hash.sort.to_h }
+ if defined?(::Ractor) && Ractor.respond_to?(:shareable_lambda)
+ sort_keys_proc = Ractor.shareable_lambda(&sort_keys_proc)
+ end
+ generator::State.default_sort_keys_proc = sort_keys_proc
+
lib/json/ext.rb
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/lib/json/ext.rb 2026-08-09 11:51:19.729162247 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/lib/json/ext.rb 2026-08-09 11:51:19.880163351 +0000
@@ -43,0 +44,26 @@
+ if defined?(ResumableParser) # Not yet available on JRuby
+ class ResumableParser
+ # Returns whether the parser is entirely done: no unconsumed bytes in
+ # the buffer, no document under construction and no parsed value
+ # awaiting retrieval.
+ #
+ # The main use case is detecting a truncated stream once the input is
+ # exhausted:
+ #
+ # loop do
+ # begin
+ # parser << socket.readpartial(4096)
+ # rescue EOFError
+ # break
+ # end
+ # while parser.parse
+ # process(parser.value)
+ # end
+ # end
+ # warn "stream was truncated" unless parser.empty?
+ def empty?
+ eos? && !partial_value? && !value?
+ end
+ end
+ end
+
lib/json/ext/generator/state.rb
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/lib/json/ext/generator/state.rb 2026-08-09 11:51:19.730162254 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/lib/json/ext/generator/state.rb 2026-08-09 11:51:19.880163351 +0000
@@ -56,0 +57 @@
+ sort_keys: sort_keys
lib/json/truffle_ruby/generator.rb
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/lib/json/truffle_ruby/generator.rb 2026-08-09 11:51:19.730162254 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/lib/json/truffle_ruby/generator.rb 2026-08-09 11:51:19.880163351 +0000
@@ -113,0 +114,2 @@
+ singleton_class.attr_accessor :default_sort_keys_proc # :nodoc:
+
@@ -166,0 +169 @@
+ @sort_keys = false
@@ -201,0 +205,27 @@
+ # Controls key sorting in the generated JSON. If set to +true+, object
+ # keys are sorted by key lexicographically. If set to a Proc, it
+ # receives the entire Hash and must return a Hash with its pairs in the
+ # desired order.
+ attr_reader :sort_keys
+
+ def sort_keys=(value) # :nodoc:
+ type_error = false
+ @sort_keys = case value
+ when Proc
+ value
+ when true
+ State.default_sort_keys_proc
+ when nil, false
+ false
+ else
+ type_error = true
+ false
+ end
+
+ if type_error
+ raise TypeError, "The `sort_keys` argument must be a boolean or a Proc"
+ end
+
+ @sort_keys
+ end
+
@@ -287,0 +318 @@
+ self.sort_keys = opts[:sort_keys] if opts.key?(:sort_keys)
@@ -352 +383 @@
- !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj)
+ !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj) and !@sort_keys
@@ -354,0 +386,4 @@
+ if @sort_keys
+ obj = @sort_keys.call(obj)
+ end
+
lib/json/version.rb
--- /tmp/d20260809-484-ff5mv2/json-2.20.0/lib/json/version.rb 2026-08-09 11:51:19.730162254 +0000
+++ /tmp/d20260809-484-ff5mv2/json-2.21.2/lib/json/version.rb 2026-08-09 11:51:19.880163351 +0000
@@ -4 +4 @@
- VERSION = '2.20.0'
+ VERSION = '2.21.2' |
Contributor
gem compare --diff json 2.20.0 2.21.2Compared versions: ["2.20.0", "2.21.2"]
DIFFERENT files:
2.20.0->2.21.2:
* Changed:
CHANGES.md
--- /tmp/d20260809-524-v1xma8/json-2.20.0/CHANGES.md 2026-08-09 11:51:26.780943632 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/CHANGES.md 2026-08-09 11:51:26.792943657 +0000
@@ -4,0 +5,16 @@
+### 2026-07-31 (2.21.2)
+
+* Fix a use-after-free bug in `JSON::ResumableParser`. [GHSA-9hj4-r449-hfvc].
+
+### 2026-07-13 (2.21.1)
+
+* Fix a compilation issue on Window and Microsoft Visual C++.
+
+### 2026-07-12 (2.21.0)
+
+* `JSON.generate` now accept a `sort_keys` option, which takes either a boolean or a block.
+* Added `#empty?` and `#partial_value?` methods on `JSON::ResumableParser`.
+* Numerous correctness and performance fixes for `JSON::ResumableParser`.
+* Avoid triggering Ruby's `float out of range` warning when parsing out of range numbers.
+* Declare C types with Ruby 4.1 `RUBY_TYPED_THREAD_SAFE_FREE`.
+
README.md
--- /tmp/d20260809-524-v1xma8/json-2.20.0/README.md 2026-08-09 11:51:26.780943632 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/README.md 2026-08-09 11:51:26.793943659 +0000
@@ -88 +88 @@
-JSON.generate(Position.new(1, 2)) # => Position not allowed in JSON (JSON::GeneratorError)
+JSON.generate(Position.new(1, 2), strict: true) # => Position not allowed in JSON (JSON::GeneratorError)
@@ -120 +120 @@
-coder = JSON::Combining.new do |object, is_object_key|
+coder = JSON::Coder.new do |object, is_object_key|
@@ -123,2 +123,2 @@
- if !string.valid_encoding? || string.encoding != Encoding::UTF_8
- Base64.encode64(string)
+ if !object.valid_encoding? || object.encoding != Encoding::UTF_8
+ Base64.encode64(object)
@@ -126 +126 @@
- string
+ object
ext/json/ext/generator/generator.c
--- /tmp/d20260809-524-v1xma8/json-2.20.0/ext/json/ext/generator/generator.c 2026-08-09 11:51:26.782943636 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/ext/json/ext/generator/generator.c 2026-08-09 11:51:26.955943998 +0000
@@ -36,0 +37 @@
+ VALUE sort_keys;
@@ -39 +40 @@
-static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8;
+static VALUE mJSON, cState, cFragment, eGeneratorError, eNestingError, Encoding_UTF_8, default_sort_keys_proc;
@@ -43 +44 @@
- sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json;
+ sym_ascii_only, sym_depth, sym_buffer_initial_length, sym_script_safe, sym_escape_slash, sym_strict, sym_as_json, sym_sort_keys;
@@ -711,0 +713 @@
+ rb_gc_mark_movable(state->sort_keys);
@@ -722,0 +725 @@
+ state->sort_keys = rb_gc_location(state->sort_keys);
@@ -742 +745 @@
- .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -771,0 +775 @@
+ RB_OBJ_WRITTEN(vstate, Qundef, state->sort_keys);
@@ -1052,0 +1057,5 @@
+ if (RB_UNLIKELY(data->state->sort_keys)) {
+ obj = rb_proc_call_with_block(data->state->sort_keys, 1, &obj, Qnil);
+ Check_Type(obj, T_HASH);
+ }
+
@@ -1378,0 +1388 @@
+ RB_OBJ_WRITTEN(obj, Qundef, objState->sort_keys);
@@ -1724,0 +1735,49 @@
+static VALUE cState_set_default_sort_keys_proc(VALUE self, VALUE proc)
+{
+ if (!rb_obj_is_proc(proc)) {
+ rb_raise(rb_eTypeError, "sort_key_proc must be a Proc");
+ }
+ return default_sort_keys_proc = proc;
+}
+
+static VALUE normalize_sort_keys(VALUE value)
+{
+ if (rb_obj_is_proc(value)) {
+ return value;
+ } else if (value == Qtrue) {
+ return default_sort_keys_proc;
+ } else if (RTEST(value)) {
+ rb_raise(rb_eTypeError, "The `sort_keys` argument must be a boolean or a Proc");
+ } else {
+ return Qfalse;
+ }
+}
+
+/*
+ * call-seq: sort_keys
+ *
+ * Get the value of sort_keys.
+ */
+static VALUE cState_sort_keys_p(VALUE self)
+{
+ GET_STATE(self);
+ return state->sort_keys;
+}
+
+/*
+ * call-seq: sort_keys=(value)
+ *
+ * value is a boolean or a proc. If the value is the boolean true, object keys
+ * will be sorted lexicographically in ascending order.
+ *
+ * If the value is a proc, it receives the entire Hash and must return a Hash
+ * with its pairs in the desired order, allowing for arbitrary sorting.
+ */
+static VALUE cState_sort_keys_set(VALUE self, VALUE value)
+{
+ rb_check_frozen(self);
+ GET_STATE(self);
+ RB_OBJ_WRITE(self, &state->sort_keys, normalize_sort_keys(value));
+ return Qnil;
+}
+
@@ -1834,0 +1894,3 @@
+ else if (key == sym_sort_keys) {
+ state_write_value(data, &state->sort_keys, normalize_sort_keys(val));
+ }
@@ -1911,0 +1974,2 @@
+ rb_global_variable(&default_sort_keys_proc);
+
@@ -1920,0 +1985,2 @@
+ rb_define_singleton_method(cState, "default_sort_keys_proc=", cState_set_default_sort_keys_proc, 1);
+
@@ -1959,0 +2026,2 @@
+ rb_define_method(cState, "sort_keys", cState_sort_keys_p, 0);
+ rb_define_method(cState, "sort_keys=", cState_sort_keys_set, 1);
@@ -1988,0 +2057 @@
+ sym_sort_keys = ID2SYM(rb_intern("sort_keys"));
ext/json/ext/json.h
--- /tmp/d20260809-524-v1xma8/json-2.20.0/ext/json/ext/json.h 2026-08-09 11:51:26.782943636 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/ext/json/ext/json.h 2026-08-09 11:51:26.956944000 +0000
@@ -33,0 +34,4 @@
+#ifndef RUBY_TYPED_THREAD_SAFE_FREE
+#define RUBY_TYPED_THREAD_SAFE_FREE RUBY_TYPED_FREE_IMMEDIATELY
+#endif
+
ext/json/ext/parser/extconf.rb
--- /tmp/d20260809-524-v1xma8/json-2.20.0/ext/json/ext/parser/extconf.rb 2026-08-09 11:51:26.782943636 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/ext/json/ext/parser/extconf.rb 2026-08-09 11:51:26.956944000 +0000
@@ -18,16 +17,0 @@
-def have_builtin_func(name, check_expr, opt = "", &b)
- checking_for checking_message(name.funcall_style, nil, opt) do
- if try_compile(<<SRC, opt, &b)
-int foo;
-int main() { #{check_expr}; return 0; }
-SRC
- $defs.push(format("-DHAVE_BUILTIN_%s", name.tr_cpp))
- true
- else
- false
- end
- end
-end
-
-have_builtin_func("__builtin_clzll", "__builtin_clzll(0)")
-
@@ -47,0 +32,6 @@
+
+# Disable function outlining on clang to prevent some of the repeated instructions
+# in json_eat_whitespace being outlined into function calls.
+# Note: This verifies '-mno-outline' is accepted as a valid compiler flag
+# and will not pass it if unsupported.
+append_cflags("-mno-outline")
ext/json/ext/parser/parser.c
--- /tmp/d20260809-524-v1xma8/json-2.20.0/ext/json/ext/parser/parser.c 2026-08-09 11:51:26.783943638 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/ext/json/ext/parser/parser.c 2026-08-09 11:51:26.956944000 +0000
@@ -8 +8,4 @@
-static ID i_new, i_try_convert, i_uminus, i_encode, i_at_line, i_at_column;
+static ID i_new, i_try_convert, i_encode, i_at_line, i_at_column;
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
+static ID i_uminus;
+#endif
@@ -317 +320 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -514 +517 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_EMBEDDABLE,
@@ -588,0 +592,2 @@
+ JSON_ASSERT(!state->parser);
+ JSON_ASSERT(state->cursor);
@@ -621,4 +626,8 @@
- long line, column;
- cursor_position(state, &line, &column);
-
- VALUE warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ VALUE warning;
+ if (state->parser) { // line and columns can't be accurate in resumable
+ warning = rb_utf8_str_new_cstr(message);
+ } else {
+ long line, column;
+ cursor_position(state, &line, &column);
+ warning = rb_sprintf("%s at line %ld column %ld", message, line, column);
+ }
@@ -768 +777 @@
-json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config)
+json_eat_comments(JSON_ParserState *state, JSON_ParserConfig *config, const char *resume_pos)
@@ -774,0 +784,5 @@
+ // An incomplete comment suspends a resumable parse by rewinding the cursor
+ // and throwing. Callers that already consumed a token not yet committed to
+ // the frame stack pass resume_pos so the rewind re-reads that token too.
+ // Non-resumable error positions keep pointing at the comment either way.
+ const char *rewind_pos = (state->parser && resume_pos) ? resume_pos : start;
@@ -779,2 +793,9 @@
- state->cursor = memchr(state->cursor, '\n', state->end - state->cursor);
- if (!state->cursor) {
+ const char *newline = memchr(state->cursor, '\n', state->end - state->cursor);
+ if (!newline) {
+ // state->parser marks resumable mode, where the buffer end is only a
+ // chunk boundary: the terminating newline may still arrive, so leave
+ // the comment unterminated instead of consuming to end as a one-shot
+ // parse would.
+ if (state->parser) {
+ raise_eos_error_at("unterminated comment, expected end of line", state, rewind_pos);
+ }
@@ -783 +804 @@
- state->cursor++;
+ state->cursor = newline + 1;
@@ -793 +814 @@
- raise_eos_error_at("unterminated comment, expected closing '*/'", state, start);
+ raise_eos_error_at("unterminated comment, expected closing '*/'", state, rewind_pos);
@@ -805 +826 @@
- raise_parse_error_at("unexpected token %s", state, start, eos(state));
+ raise_parse_error_at("unexpected token %s", state, eos(state) ? rewind_pos : start, eos(state));
@@ -816 +837 @@
-json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+json_eat_whitespace_resume_at(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments, const char *resume_pos)
@@ -851 +872 @@
- json_eat_comments(state, config);
+ json_eat_comments(state, config, resume_pos);
@@ -859,0 +881,6 @@
+ALWAYS_INLINE(static) void
+json_eat_whitespace(JSON_ParserState *state, JSON_ParserConfig *config, bool include_comments)
+{
+ json_eat_whitespace_resume_at(state, config, include_comments, NULL);
+}
+
@@ -1132,0 +1160,7 @@
+ // If the value is so small that it definitely underflows to 0.0, return early
+ // to avoid triggering a "Float out of range" warning from rb_cstr_to_dbl.
+ // When mantissa_digits + exponent < -324, value < 10^(-324) < DBL_TRUE_MIN/2,
+ // so it rounds to 0 in IEEE 754 round-to-nearest.
+ if (RB_UNLIKELY(mantissa_digits + exponent < -324)) {
+ return rb_float_new(negative ? -0.0 : 0.0);
+ }
@@ -1432 +1466 @@
-static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start)
+static inline VALUE json_parse_number(JSON_ParserState *state, JSON_ParserConfig *config, bool negative, const char *start, bool resumable)
@@ -1488,0 +1523,10 @@
+ // A number touching the end of the buffer may still grow in a later chunk,
+ // so the caller will rewind and wait. Decoding it now would build a value
+ // -- for a long run of digits, an expensive bignum -- only to discard it,
+ // and repeating that on every resumed chunk is quadratic in the number's
+ // length. The digit scan above already advanced the cursor, which is all
+ // the caller needs to detect the incomplete number.
+ if (RB_UNLIKELY(resumable && eos(state))) {
+ return Qundef;
+ }
+
@@ -1565,0 +1610,7 @@
+ // A trailing comma lands us here expecting an element but finding the
+ // closing bracket; hand off to ARRAY_COMMA to close. An empty array
+ // closes inline at '[', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && frame->type == JSON_FRAME_ARRAY && peek(state) == ']') {
+ goto JSON_PHASE_ARRAY_COMMA;
+ }
+
@@ -1606 +1657 @@
- value = json_parse_number(state, config, true, value_start);
+ value = json_parse_number(state, config, true, value_start, resumable);
@@ -1629 +1680 @@
- value = json_parse_number(state, config, false, value_start);
+ value = json_parse_number(state, config, false, value_start, resumable);
@@ -1661 +1712,3 @@
- json_eat_whitespace(state, config, true);
+ // The '[' is consumed but its frame is only pushed below, so a
+ // comment suspending here must resume from the bracket.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1690 +1743,2 @@
- json_eat_whitespace(state, config, true);
+ // Same as '[': the frame is only pushed below.
+ json_eat_whitespace_resume_at(state, config, true, value_start);
@@ -1747,0 +1802,7 @@
+ // A trailing comma lands us here expecting a key but finding the closing
+ // brace; hand off to OBJECT_COMMA to close. An empty object closes inline
+ // at '{', so this position is only reached after a ','.
+ if (config->allow_trailing_comma && peek(state) == '}') {
+ goto JSON_PHASE_OBJECT_COMMA;
+ }
+
@@ -1810,7 +1871,4 @@
- if (config->allow_trailing_comma) {
- json_eat_whitespace(state, config, true);
- if (peek(state) == ']') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_ARRAY_COMMA;
- }
- }
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_VALUE once the ']' is in the buffer.
@@ -1855,9 +1913,4 @@
- json_eat_whitespace(state, config, true);
-
- if (config->allow_trailing_comma) {
- if (peek(state) == '}') {
- // Trailing comma: stay in COMMA to close on the next iteration.
- goto JSON_PHASE_OBJECT_COMMA;
- }
- }
-
+ // Commit the phase before eating the whitespace that follows: an
+ // incomplete comment there would suspend the parse, and a phase not
+ // yet advanced past the ',' would drop it on resume. A trailing comma
+ // is recognized in JSON_PHASE_OBJECT_KEY once the '}' is in the buffer.
@@ -2183 +2236 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_WB_PROTECTED | RUBY_TYPED_FROZEN_SHAREABLE | RUBY_TYPED_EMBEDDABLE,
@@ -2268 +2321 @@
- .flags = RUBY_TYPED_FREE_IMMEDIATELY | RUBY_TYPED_EMBEDDABLE,
+ .flags = RUBY_TYPED_THREAD_SAFE_FREE | RUBY_TYPED_EMBEDDABLE,
@@ -2521,0 +2575 @@
+ parser->state.start = parser->state.cursor = parser->state.end = 0;
@@ -2618 +2672,6 @@
- // Copy the value stack as we need to mutate it.
+ // Copy the value stack as we need to mutate it. The collapse loop folds each
+ // open container by popping its entries and pushing the single result, so a
+ // parent always reclaims its child's slot; head exceeds its live size by at
+ // most one, either for the missing-value placeholder pushed below or for the
+ // result of folding an empty innermost container. That one spare slot keeps
+ // rvalue_stack_push from growing (reallocating) this ALLOCV buffer.
@@ -2620,3 +2679,3 @@
- parser.value_stack.capa = (capa + missing_object_value);
- VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, capa + missing_object_value);
- MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, parser.value_stack.capa);
+ parser.value_stack.capa = capa + 1;
+ VALUE tmpbuf, *value_stack_buffer = ALLOCV_N(VALUE, tmpbuf, parser.value_stack.capa);
+ MEMCPY(value_stack_buffer, parser.value_stack.ptr, VALUE, capa);
@@ -2715 +2774 @@
- * call-seq: value? -> true or false
+ * call-seq: eos? -> true or false
@@ -2725,0 +2785,35 @@
+ * call-seq: partial_value? -> true or false
+ *
+ * Returns whether a document is currently under construction: an unclosed
+ * container, a key awaiting its value, etc.
+ *
+ * It answers the same question as <tt>!partial_value.nil?</tt>, but as a
+ * cheap predicate on the parser's internal state, without materializing the
+ * partially parsed Ruby objects:
+ * parser << '{"a":1,'
+ * parser.parse # => false
+ * parser.partial_value? # => true
+ *
+ * A fully parsed document whose value hasn't been retrieved yet is not under
+ * construction: #value? returns true and #partial_value? returns false.
+ */
+static VALUE cResumableParser_partial_value_p(VALUE self)
+{
+ JSON_ResumableParser *parser = cResumableParser_get(self);
+
+ // Mirror of #value?: values on the stack while the document isn't DONE
+ // belong to a partially built document. A container whose first key or
+ // element hasn't been parsed yet has no frame nor value registered (the
+ // tokenizer rewinds to the container start on EOS), so that state is
+ // observable through the buffer (#eos?/#rest) instead, keeping this
+ // predicate consistent with #partial_value returning nil.
+ if (parser->value_stack.head > 0) {
+ json_frame *frame = json_frame_stack_peek(&parser->frames);
+ if (frame->phase != JSON_PHASE_DONE) {
+ return Qtrue;
+ }
+ }
+ return Qfalse;
+}
+
+/*
@@ -2780,0 +2875 @@
+ rb_define_method(cResumableParser, "partial_value?", cResumableParser_partial_value_p, 0);
@@ -2814,0 +2910 @@
+#ifndef HAVE_RB_STR_TO_INTERNED_STR
@@ -2815,0 +2912 @@
+#endif
lib/json.rb
--- /tmp/d20260809-524-v1xma8/json-2.20.0/lib/json.rb 2026-08-09 11:51:26.786943644 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/lib/json.rb 2026-08-09 11:51:26.958944004 +0000
@@ -124 +124,3 @@
-# defaults to +100+; specify +false+ to disable depth checking.
+# defaults to +100+;
+# You can set it to +false+ to disable depth checking entirely, but that is dangerous
+# when parsing untrusted input.
@@ -126 +128 @@
-# With the default, +false+:
+# With the default, +100+:
@@ -386,0 +389,9 @@
+# With +false+:
+# obj = []
+# obj[0] = obj
+# # Raises SystemStackError: stack level too deep
+# JSON.generate(obj, max_nesting: false)
+#
+# Setting +max_nesting+ to +false+ can lead to a stackoverflow and may leave the program
+# in an unrecoverable state. It is discouraged.
+#
@@ -411 +421,0 @@
-# defaults to the empty \String, <tt>''</tt>;
@@ -418,0 +429,5 @@
+# - Option +sort_keys+ (boolean or \Proc) controls whether and how the keys of a
+# hash are sorted when generating the output; defaults to <tt>false</tt>.
+# When +true+, keys are sorted lexicographically. When a \Proc, it receives
+# the entire \Hash and must return a \Hash with its pairs in the desired
+# order, allowing for arbitrary sort orders.
lib/json/common.rb
--- /tmp/d20260809-524-v1xma8/json-2.20.0/lib/json/common.rb 2026-08-09 11:51:26.789943651 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/lib/json/common.rb 2026-08-09 11:51:26.960944008 +0000
@@ -157,0 +158,9 @@
+
+ # The default proc used when the +sort_keys+ generation option is +true+.
+ # It returns a new hash with the entries sorted by their keys.
+ sort_keys_proc = ->(hash) { hash.sort.to_h }
+ if defined?(::Ractor) && Ractor.respond_to?(:shareable_lambda)
+ sort_keys_proc = Ractor.shareable_lambda(&sort_keys_proc)
+ end
+ generator::State.default_sort_keys_proc = sort_keys_proc
+
lib/json/ext.rb
--- /tmp/d20260809-524-v1xma8/json-2.20.0/lib/json/ext.rb 2026-08-09 11:51:26.790943653 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/lib/json/ext.rb 2026-08-09 11:51:26.960944008 +0000
@@ -43,0 +44,26 @@
+ if defined?(ResumableParser) # Not yet available on JRuby
+ class ResumableParser
+ # Returns whether the parser is entirely done: no unconsumed bytes in
+ # the buffer, no document under construction and no parsed value
+ # awaiting retrieval.
+ #
+ # The main use case is detecting a truncated stream once the input is
+ # exhausted:
+ #
+ # loop do
+ # begin
+ # parser << socket.readpartial(4096)
+ # rescue EOFError
+ # break
+ # end
+ # while parser.parse
+ # process(parser.value)
+ # end
+ # end
+ # warn "stream was truncated" unless parser.empty?
+ def empty?
+ eos? && !partial_value? && !value?
+ end
+ end
+ end
+
lib/json/ext/generator/state.rb
--- /tmp/d20260809-524-v1xma8/json-2.20.0/lib/json/ext/generator/state.rb 2026-08-09 11:51:26.790943653 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/lib/json/ext/generator/state.rb 2026-08-09 11:51:26.961944011 +0000
@@ -56,0 +57 @@
+ sort_keys: sort_keys
lib/json/truffle_ruby/generator.rb
--- /tmp/d20260809-524-v1xma8/json-2.20.0/lib/json/truffle_ruby/generator.rb 2026-08-09 11:51:26.791943655 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/lib/json/truffle_ruby/generator.rb 2026-08-09 11:51:26.961944011 +0000
@@ -113,0 +114,2 @@
+ singleton_class.attr_accessor :default_sort_keys_proc # :nodoc:
+
@@ -166,0 +169 @@
+ @sort_keys = false
@@ -201,0 +205,27 @@
+ # Controls key sorting in the generated JSON. If set to +true+, object
+ # keys are sorted by key lexicographically. If set to a Proc, it
+ # receives the entire Hash and must return a Hash with its pairs in the
+ # desired order.
+ attr_reader :sort_keys
+
+ def sort_keys=(value) # :nodoc:
+ type_error = false
+ @sort_keys = case value
+ when Proc
+ value
+ when true
+ State.default_sort_keys_proc
+ when nil, false
+ false
+ else
+ type_error = true
+ false
+ end
+
+ if type_error
+ raise TypeError, "The `sort_keys` argument must be a boolean or a Proc"
+ end
+
+ @sort_keys
+ end
+
@@ -287,0 +318 @@
+ self.sort_keys = opts[:sort_keys] if opts.key?(:sort_keys)
@@ -352 +383 @@
- !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj)
+ !@ascii_only and !@script_safe and @max_nesting == 0 and (!@strict || Symbol === obj) and !@sort_keys
@@ -354,0 +386,4 @@
+ if @sort_keys
+ obj = @sort_keys.call(obj)
+ end
+
lib/json/version.rb
--- /tmp/d20260809-524-v1xma8/json-2.20.0/lib/json/version.rb 2026-08-09 11:51:26.791943655 +0000
+++ /tmp/d20260809-524-v1xma8/json-2.21.2/lib/json/version.rb 2026-08-09 11:51:26.961944011 +0000
@@ -4 +4 @@
- VERSION = '2.20.0'
+ VERSION = '2.21.2' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps json from 2.20.0 to 2.21.2.
Release notes
Sourced from json's releases.
Changelog
Sourced from json's changelog.
Commits
5a32e43Release 2.21.22c332bfAlso don't compute ResumableParser cursor position for warning5499aa5Improve max_nesting documentation499d9bcFix the parser benchmark bytes reportf5fc245Pass '-mno-outline' to the compiler (if supported) to prevent function outlin...fd61defRelease 2.21.1db70b14Rely on Ruby's HAVE_BUILTIN___BUILTIN_CLZLL9be0e8eRelease 2.21.0d1299d4Avoid re-decoding an incomplete number on every ResumableParser chunkcc010c4Fix ResumableParser losing tokens before a feed-boundary suspensionDependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting
@dependabot rebase.Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR:
@dependabot rebasewill rebase this PR@dependabot recreatewill recreate this PR, overwriting any edits that have been made to it@dependabot show <dependency name> ignore conditionswill show all of the ignore conditions of the specified dependency@dependabot ignore this major versionwill close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this minor versionwill close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)@dependabot ignore this dependencywill close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)You can disable automated security fix PRs for this repo from the Security Alerts page.