Skip to content

Commit 5850976

Browse files
aycabtanobuk0kubunno6vko1
authored
Backport lib/reline, ext/readline, and lib/irb for 3.0.1 (#4085)
* Get rid of inconsistent dll linkages against vcpkg readline * [ruby/irb] Enhance colored inspect output ruby/irb@dffcdb5269 * [ruby/irb] Add color_printer.rb to gemspec ruby/irb@b4df0fd8b2 * [ruby/irb] Fix failing tests ruby/irb@7723ade899 * irb: add more syntax errors colorizing support (#3967) * [ruby/irb] Do not colorize partially-correct inspect This is to prevent a yellow-mixed output for ActiveSupport::TimeWithZone. Follows up ruby/irb#159 and #3967. ruby/irb@a5804c3 * [ruby/irb] Remove unnecessary ignore_error in dispatch_seq Just forgotten in ruby/irb@a5804c3 ruby/irb@e42e548793 * Increase timeout for reline with --jit-wait for failures like: http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201229-130509 http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201229-165132 http://ci.rvm.jp/logfiles/brlog.trunk-mjit-wait.20201228-015519 * [ruby/irb] Stringify when a non-object is passed to PP#text If a nested object is passed to #pp, it may be sometimes passed to the #text method as an object without being stringified. This is fixed on the Ruby main repository; 433a3be but it was a bug of Ripper so still needs this workaround for using irb as a gem on Ruby 3.0.0 or earlier. Co-authored-by: k0kubun <[email protected]> ruby/irb@8d13df22ee * [ruby/irb] Newline in oneliner def doesn't reset indent This closes ruby/irb#132. ruby/irb@43456dcf5e * [ruby/irb] Escape invalid byte sequence in Exception This fixes ruby/irb#141. ruby/irb@0815317d42 * [ruby/irb] Handle indentations related to keyword "do" correctly This fixes ruby/irb#158. ruby/irb@964643400b * [ruby/irb] Heredoc may contain multiple newlines in a single token Use the start token as the indentation criteria so that it works properly in heredoc. ref. ruby/reline#242 ruby/irb@9704808dfd * [ruby/irb] Use Ripper::Lexer#scan to take broken tokens ref. ruby/reline#242 ruby/irb@54f90cb6c9 * [ruby/irb] Use error tokens if there are no correct tokens in the same place For example, the broken code "%www" will result in only one error token. ruby/irb@9fa39a7cf3 * [ruby/irb] Ensure to restore $VERBOSE ruby/irb@cef474a76a * 600x larger timeout for Reline I didn't notice it's msec. 2.5s is too short. http://ci.rvm.jp/results/trunk-mjit-wait@phosphorus-docker/3311385 * [ruby/irb] fix typo in `IRB::Irb#convert_invalid_byte_sequence` ruby/irb@d09d3c3d68 * [ruby/irb] do not escape a predicate method for doc namespace * Fixes #88 ruby/irb@d431a30af4 * [ruby/irb] refactoring an error handling in `IRB::Inspector` * moved rescue clause to `#inspect_value` to catch all failures in inspectors * test with all (currently five kind of) inspect modes - tweaked the input due to only `Marshal` can inspect(dump) a `BasicObject` ruby/irb@9d112fab8e * [ruby/irb] Use Exception#full_message to show backtrace in the correct order [Bug #17466] ruby/irb@1c76845cca * [ruby/irb] Fix BACK_TRACE_LIMIT logic ruby/irb@30dc5d43fe * irb: Drop lines from backtrace for tests in Ruby repository * [ruby/reline] Update cursor correctly when just cursor moving This fixes ruby/reline#236 and ruby/reline#239. ruby/reline@3e3c89d00b * [ruby/reline] Correct var names in Reline were different from vi-*-mode-string ruby/reline@8255fc93b9 * [ruby/reline] Remove debug print ruby/reline@d7fbaedc6a * [ruby/reline] Suppress crashing when auto_indent_proc returns broken indent info Co-authored-by: Juanito Fatas <[email protected]> ruby/reline@7c24276275 * [ruby/reline] Suppress crashing when dynamic_prompt_proc returns a broken prompt list Co-authored-by: Juanito Fatas <[email protected]> ruby/reline@558f7be168 * [ruby/reline] Suppress auto indent for adding newlines in pasting Co-authored-by: Juanito Fatas <[email protected]> ruby/reline@074bb017a7 * [ruby/reline] Add acknowledgments and license for rb-readline ruby/reline@19df59b916 * [ruby/irb] Fix comment, irb gem supports 2.5.0 or older ruby/irb@36118015ba * should use `assert_include` here. Random ordering test can introduce antoher candidate so it should be `assert_include`. * [ruby/irb] Add missing require This is useful if you want to use IRB::ColorPrinter as a library like: ``` begin require 'irb/color_printer' IRB::ColorPrinter.pp(obj) rescue LoadError pp(obj) end ``` ruby/irb@f8461691c7 * [ruby/irb] Make IRB::ColorPrinter.pp compatible with PP.pp The incompatible interface is not helpful, again if you want to use it as a standalone library, falling it back to PP. Original PP.pp also ends with `out << "\n"`. ruby/irb@4c74c7d84c * Suppress constant redefinition warnings * Fix the failing test with XDG_CONFIG_HOME * [ruby/irb] Version 1.3.1 ruby/irb@c230d08911 * [ruby/reline] Handle ed_search_{prev,next}_history in multiline correctly The current line was being handled incorrectly when displaying the hit history, so it has been fixed to be correct. ruby/reline@a3df4343b3 * [ruby/reline] Move the cursor correctly when deleting at eol This fixes ruby/reline#246. ruby/reline@07a73ba601 * [ruby/reline] Version 0.2.1 ruby/reline@a3b3c6ee60 * [ruby/reline] Initialize a variable just in case ruby/reline@29b10f6e98 * [ruby/reline] Tests with yamatanooroti don't need chdir Because of chdir, log files ware created in temporary directries on Windows. ruby/reline@200b469a68 * [ruby/reline] Windows needs more times to wait rendering ruby/reline@53ff2b09c7 * [ruby/reline] Support for change in Windows-specific behavior at eol The behavior of automatically moving the cursor to the next line when displaying a char at the eol on Windows suddenly disappeared. ruby/reline@cad4de6ee8 * [ruby/reline] Reline::Windows.erase_after_cursor erases attributes too ruby/reline@68b961dfc7 * [ruby/irb] [ruby/irb] [ruby/reline] Version 0.2.2 ruby/reline@dfb710946f ruby/irb@1a1cdf9628 ruby/irb@fe99faf8bd * [ruby/irb] handle `__ENCODING__` as a keyword as well ruby/irb@a6a33d908f * [ruby/irb] handle repeated exception separately ruby/irb@fcf6b34bc5 * [ruby/irb] skip a failling test on TruffleRuby * due to the difference of backtrace pointed out by @aycabta ruby/irb@5e00a0ae61 * [ruby/irb] Version 1.3.2 ruby/irb@a7699026cc Co-authored-by: Nobuyoshi Nakada <[email protected]> Co-authored-by: Takashi Kokubun <[email protected]> Co-authored-by: Nobuhiro IMAI <[email protected]> Co-authored-by: Koichi Sasada <[email protected]> Co-authored-by: Hiroshi SHIBATA <[email protected]>
1 parent 29777cb commit 5850976

26 files changed

+812
-163
lines changed

‎ext/readline/readline.c‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -78,7 +78,7 @@ static ID id_special_prefixes;
7878
#ifndef HAVE_RL_USERNAME_COMPLETION_FUNCTION
7979
# define rl_username_completion_function username_completion_function
8080
#else
81-
char *rl_username_completion_function(const char *, int);
81+
RUBY_EXTERN char *rl_username_completion_function(const char *, int);
8282
#endif
8383
#ifndef HAVE_RL_COMPLETION_MATCHES
8484
# define rl_completion_matches completion_matches
@@ -689,7 +689,7 @@ readline_s_insert_text(VALUE self, VALUE str)
689689
#endif
690690

691691
#if defined(HAVE_RL_DELETE_TEXT)
692-
int rl_delete_text(int, int);
692+
RUBY_EXTERN int rl_delete_text(int, int);
693693
static const char *
694694
str_subpos(const char *ptr, const char *end, long beg, long *sublen, rb_encoding *enc)
695695
{
@@ -1148,7 +1148,7 @@ readline_s_get_screen_size(VALUE self)
11481148
#endif
11491149

11501150
#ifdef HAVE_RL_VI_EDITING_MODE
1151-
int rl_vi_editing_mode(int, int);
1151+
RUBY_EXTERN int rl_vi_editing_mode(int, int);
11521152
/*
11531153
* call-seq:
11541154
* Readline.vi_editing_mode -> nil
@@ -1187,7 +1187,7 @@ readline_s_vi_editing_mode_p(VALUE self)
11871187
#endif
11881188

11891189
#ifdef HAVE_RL_EMACS_EDITING_MODE
1190-
int rl_emacs_editing_mode(int, int);
1190+
RUBY_EXTERN int rl_emacs_editing_mode(int, int);
11911191
/*
11921192
* call-seq:
11931193
* Readline.emacs_editing_mode -> nil
@@ -1672,7 +1672,7 @@ readline_s_get_filename_quote_characters(VALUE self)
16721672
#endif
16731673

16741674
#ifdef HAVE_RL_REFRESH_LINE
1675-
int rl_refresh_line(int, int);
1675+
RUBY_EXTERN int rl_refresh_line(int, int);
16761676
/*
16771677
* call-seq:
16781678
* Readline.refresh_line -> nil
@@ -1919,7 +1919,7 @@ username_completion_proc_call(VALUE self, VALUE str)
19191919
}
19201920

19211921
#ifdef HAVE_RL_CLEAR_SIGNALS
1922-
int rl_clear_signals(void);
1922+
RUBY_EXTERN int rl_clear_signals(void);
19231923
#endif
19241924

19251925
#undef rb_intern

‎lib/irb.rb‎

Lines changed: 59 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -574,10 +574,35 @@ def eval_input
574574
next
575575
end
576576
handle_exception(exc)
577+
@context.workspace.local_variable_set(:_, exc)
578+
exc = nil
577579
end
578580
end
579581
end
580582

583+
def convert_invalid_byte_sequence(str)
584+
str = str.force_encoding(Encoding::ASCII_8BIT)
585+
conv = Encoding::Converter.new(Encoding::ASCII_8BIT, Encoding::UTF_8)
586+
dst = String.new
587+
begin
588+
ret = conv.primitive_convert(str, dst)
589+
case ret
590+
when :invalid_byte_sequence
591+
conv.insert_output(conf.primitive_errinfo[3].dump[1..-2])
592+
redo
593+
when :undefined_conversion
594+
c = conv.primitive_errinfo[3].dup.force_encoding(conv.primitive_errinfo[1])
595+
conv.insert_output(c.dump[1..-2])
596+
redo
597+
when :incomplete_input
598+
conv.insert_output(conv.primitive_errinfo[3].dump[1..-2])
599+
when :finished
600+
end
601+
break
602+
end while nil
603+
dst
604+
end
605+
581606
def handle_exception(exc)
582607
if exc.backtrace && exc.backtrace[0] =~ /\/irb(2)?(\/.*|-.*|\.rb)?:/ && exc.class.to_s !~ /^IRB/ &&
583608
!(SyntaxError === exc) && !(EncodingError === exc)
@@ -587,49 +612,44 @@ def handle_exception(exc)
587612
irb_bug = false
588613
end
589614

590-
if STDOUT.tty?
591-
attr = ATTR_TTY
592-
print "#{attr[1]}Traceback#{attr[]} (most recent call last):\n"
593-
else
594-
attr = ATTR_PLAIN
595-
end
596-
messages = []
597-
lasts = []
598-
levels = 0
599615
if exc.backtrace
600-
count = 0
601-
exc.backtrace.each do |m|
602-
m = @context.workspace.filter_backtrace(m) or next unless irb_bug
603-
count += 1
604-
if attr == ATTR_TTY
605-
m = sprintf("%9d: from %s", count, m)
616+
order = nil
617+
if '2.5.0' == RUBY_VERSION
618+
# Exception#full_message doesn't have keyword arguments.
619+
message = exc.full_message # the same of (highlight: true, order: bottom)
620+
order = :bottom
621+
elsif '2.5.1' <= RUBY_VERSION && RUBY_VERSION < '3.0.0'
622+
if STDOUT.tty?
623+
message = exc.full_message(order: :bottom)
624+
order = :bottom
606625
else
607-
m = "\tfrom #{m}"
608-
end
609-
if messages.size < @context.back_trace_limit
610-
messages.push(m)
611-
elsif lasts.size < @context.back_trace_limit
612-
lasts.push(m).shift
613-
levels += 1
626+
message = exc.full_message(order: :top)
627+
order = :top
614628
end
629+
else # '3.0.0' <= RUBY_VERSION
630+
message = exc.full_message(order: :top)
631+
order = :top
615632
end
616-
end
617-
if attr == ATTR_TTY
618-
unless lasts.empty?
619-
puts lasts.reverse
620-
printf "... %d levels...\n", levels if levels > 0
621-
end
622-
puts messages.reverse
623-
end
624-
m = exc.to_s.split(/\n/)
625-
print "#{attr[1]}#{exc.class} (#{attr[4]}#{m.shift}#{attr[0, 1]})#{attr[]}\n"
626-
puts m.map {|s| "#{attr[1]}#{s}#{attr[]}\n"}
627-
if attr == ATTR_PLAIN
628-
puts messages
629-
unless lasts.empty?
630-
puts lasts
631-
printf "... %d levels...\n", levels if levels > 0
632-
end
633+
message = convert_invalid_byte_sequence(message)
634+
message = message.gsub(/((?:^\t.+$\n)+)/) { |m|
635+
case order
636+
when :top
637+
lines = m.split("\n")
638+
when :bottom
639+
lines = m.split("\n").reverse
640+
end
641+
unless irb_bug
642+
lines = lines.map { |l| @context.workspace.filter_backtrace(l) }.compact
643+
if lines.size > @context.back_trace_limit
644+
omit = lines.size - @context.back_trace_limit
645+
lines = lines[0..(@context.back_trace_limit - 1)]
646+
lines << "\t... %d levels..." % omit
647+
end
648+
end
649+
lines = lines.reverse if order == :bottom
650+
lines.map{ |l| l + "\n" }.join
651+
}
652+
puts message
633653
end
634654
print "Maybe IRB bug!\n" if irb_bug
635655
end

‎lib/irb/color.rb‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ module Color
1717
CYAN = 36
1818

1919
TOKEN_KEYWORDS = {
20-
on_kw: ['nil', 'self', 'true', 'false', '__FILE__', '__LINE__'],
20+
on_kw: ['nil', 'self', 'true', 'false', '__FILE__', '__LINE__', '__ENCODING__'],
2121
on_const: ['ENV'],
2222
}
2323
private_constant :TOKEN_KEYWORDS
@@ -60,13 +60,20 @@ module Color
6060
on_words_beg: [[RED, BOLD], ALL],
6161
on_parse_error: [[RED, REVERSE], ALL],
6262
compile_error: [[RED, REVERSE], ALL],
63+
on_assign_error: [[RED, REVERSE], ALL],
64+
on_alias_error: [[RED, REVERSE], ALL],
65+
on_class_name_error:[[RED, REVERSE], ALL],
66+
on_param_error: [[RED, REVERSE], ALL],
6367
}
6468
rescue NameError
6569
# Give up highlighting Ripper-incompatible older Ruby
6670
TOKEN_SEQ_EXPRS = {}
6771
end
6872
private_constant :TOKEN_SEQ_EXPRS
6973

74+
ERROR_TOKENS = TOKEN_SEQ_EXPRS.keys.select { |k| k.to_s.end_with?('error') }
75+
private_constant :ERROR_TOKENS
76+
7077
class << self
7178
def colorable?
7279
$stdout.tty? && supported? && (/mswin|mingw/ =~ RUBY_PLATFORM || (ENV.key?('TERM') && ENV['TERM'] != 'dumb'))
@@ -107,14 +114,19 @@ def colorize(text, seq)
107114
# If `complete` is false (code is incomplete), this does not warn compile_error.
108115
# This option is needed to avoid warning a user when the compile_error is happening
109116
# because the input is not wrong but just incomplete.
110-
def colorize_code(code, complete: true)
117+
def colorize_code(code, complete: true, ignore_error: false)
111118
return code unless colorable?
112119

113120
symbol_state = SymbolState.new
114121
colored = +''
115122
length = 0
116123

117124
scan(code, allow_last_error: !complete) do |token, str, expr|
125+
# IRB::ColorPrinter skips colorizing fragments with any invalid token
126+
if ignore_error && ERROR_TOKENS.include?(token)
127+
return Reline::Unicode.escape_for_print(code)
128+
end
129+
118130
in_symbol = symbol_state.scan_token(token)
119131
str.each_line do |line|
120132
line = Reline::Unicode.escape_for_print(line)
@@ -180,11 +192,12 @@ def scan(code, allow_last_error:)
180192
end
181193
end
182194
end
195+
ensure
183196
$VERBOSE = verbose
184197
end
185198

186199
def dispatch_seq(token, expr, str, in_symbol:)
187-
if token == :on_parse_error or token == :compile_error
200+
if ERROR_TOKENS.include?(token)
188201
TOKEN_SEQ_EXPRS[token][0]
189202
elsif in_symbol
190203
[YELLOW]

‎lib/irb/color_printer.rb‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# frozen_string_literal: true
2+
require 'pp'
3+
require 'irb/color'
4+
5+
module IRB
6+
class ColorPrinter < ::PP
7+
def self.pp(obj, out = $>, width = 79)
8+
q = ColorPrinter.new(out, width)
9+
q.guard_inspect_key {q.pp obj}
10+
q.flush
11+
out << "\n"
12+
end
13+
14+
def text(str, width = nil)
15+
unless str.is_a?(String)
16+
str = str.inspect
17+
end
18+
width ||= str.length
19+
20+
case str
21+
when /\A#</, '=', '>'
22+
super(Color.colorize(str, [:GREEN]), width)
23+
else
24+
super(Color.colorize_code(str, ignore_error: true), width)
25+
end
26+
end
27+
end
28+
end

‎lib/irb/completion.rb‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
4747
when /^((["'`]).*\2)\.([^.]*)$/
4848
# String
4949
receiver = $1
50-
message = Regexp.quote($3)
50+
message = $3
5151

5252
candidates = String.instance_methods.collect{|m| m.to_s}
5353
if doc_namespace
@@ -59,7 +59,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
5959
when /^(\/[^\/]*\/)\.([^.]*)$/
6060
# Regexp
6161
receiver = $1
62-
message = Regexp.quote($2)
62+
message = $2
6363

6464
candidates = Regexp.instance_methods.collect{|m| m.to_s}
6565
if doc_namespace
@@ -71,7 +71,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
7171
when /^([^\]]*\])\.([^.]*)$/
7272
# Array
7373
receiver = $1
74-
message = Regexp.quote($2)
74+
message = $2
7575

7676
candidates = Array.instance_methods.collect{|m| m.to_s}
7777
if doc_namespace
@@ -83,7 +83,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
8383
when /^([^\}]*\})\.([^.]*)$/
8484
# Proc or Hash
8585
receiver = $1
86-
message = Regexp.quote($2)
86+
message = $2
8787

8888
proc_candidates = Proc.instance_methods.collect{|m| m.to_s}
8989
hash_candidates = Hash.instance_methods.collect{|m| m.to_s}
@@ -117,7 +117,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
117117
when /^([A-Z].*)::([^:.]*)$/
118118
# Constant or class methods
119119
receiver = $1
120-
message = Regexp.quote($2)
120+
message = $2
121121
begin
122122
candidates = eval("#{receiver}.constants.collect{|m| m.to_s}", bind)
123123
candidates |= eval("#{receiver}.methods.collect{|m| m.to_s}", bind)
@@ -134,7 +134,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
134134
# Symbol
135135
receiver = $1
136136
sep = $2
137-
message = Regexp.quote($3)
137+
message = $3
138138

139139
candidates = Symbol.instance_methods.collect{|m| m.to_s}
140140
if doc_namespace
@@ -147,7 +147,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
147147
# Numeric
148148
receiver = $~[:num]
149149
sep = $~[:sep]
150-
message = Regexp.quote($~[:mes])
150+
message = $~[:mes]
151151

152152
begin
153153
instance = eval(receiver, bind)
@@ -169,7 +169,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
169169
# Numeric(0xFFFF)
170170
receiver = $1
171171
sep = $2
172-
message = Regexp.quote($3)
172+
message = $3
173173

174174
begin
175175
instance = eval(receiver, bind)
@@ -201,7 +201,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
201201
# variable.func or func.func
202202
receiver = $1
203203
sep = $2
204-
message = Regexp.quote($3)
204+
message = $3
205205

206206
gv = eval("global_variables", bind).collect{|m| m.to_s}.push("true", "false", "nil")
207207
lv = eval("local_variables", bind).collect{|m| m.to_s}
@@ -244,7 +244,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
244244
# unknown(maybe String)
245245

246246
receiver = ""
247-
message = Regexp.quote($1)
247+
message = $1
248248

249249
candidates = String.instance_methods(true).collect{|m| m.to_s}
250250
if doc_namespace
@@ -294,7 +294,7 @@ def self.retrieve_completion_data(input, bind: IRB.conf[:MAIN_CONTEXT].workspace
294294
Operators = %w[% & * ** + - / < << <= <=> == === =~ > >= >> [] []= ^ ! != !~]
295295

296296
def self.select_message(receiver, message, candidates, sep = ".")
297-
candidates.grep(/^#{message}/).collect do |e|
297+
candidates.grep(/^#{Regexp.quote(message)}/).collect do |e|
298298
case e
299299
when /^[a-zA-Z_]/
300300
receiver + sep + e

0 commit comments

Comments
 (0)