Skip to content

Commit a8df5c5

Browse files
authored
Parse RBS files as RDoc input (#1728)
## Problem RDoc already supports parser classes that register handled file names through [`parse_files_matching`](https://ruby.github.io/rdoc/RDoc/Parser.html#method-c-parse_files_matching), but `.rbs` documentation parsing currently comes from the `rbs` gem's `rdoc/discover` hook, which defines `RDoc::Parser::RBS` and delegates to `RBS::RDocPlugin::Parser` at runtime ([discovery hook](https://github.com/ruby/rbs/blob/master/lib/rdoc/discover.rb), [plugin parser](https://github.com/ruby/rbs/blob/master/lib/rdoc_plugin/parser.rb)). That leaves RBS documentation generation coupled to a separately released plugin even though RDoc already uses RBS for type-signature display ([#1665](#1665)), and internal RDoc API changes can break the plugin independently of RDoc releases ([#1711](#1711)). ## Solution - Add `RDoc::Parser::RBS` inside RDoc, adapted from the existing RBS plugin parser, so `.rbs` files are registered as RDoc input and common RBS declarations/members become RDoc code objects ([RBS plugin parser](https://github.com/ruby/rbs/blob/master/lib/rdoc_plugin/parser.rb), [PR diff](https://github.com/ruby/rdoc/pull/1728/files)). - Extend existing Ruby-source documentation when matching RBS declarations are parsed, while still generating documentation for classes, modules, methods, attributes, and constants that only appear in RBS ([PR diff](https://github.com/ruby/rdoc/pull/1728/files)). - Keep RDoc's existing `sig/**/*.rbs` type-signature loading path and skip the released `rbs` discovery hook when RDoc's parser is available, so RubyGems discovery does not replace the in-tree parser ([#1665](#1665), [discovery hook](https://github.com/ruby/rbs/blob/master/lib/rdoc/discover.rb), [PR diff](https://github.com/ruby/rdoc/pull/1728/files)).
1 parent f89b93a commit a8df5c5

12 files changed

Lines changed: 1005 additions & 144 deletions

File tree

AGENTS.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Project Overview
44

5-
**RDoc** is Ruby's default documentation generation tool that produces HTML and command-line documentation for Ruby projects. It parses Ruby source code, C extensions, and markup files to generate documentation.
5+
**RDoc** is Ruby's default documentation generation tool that produces HTML and command-line documentation for Ruby projects. It parses Ruby source code, C extensions, RBS signature files, and markup files to generate documentation.
66

77
- **Repository:** https://github.com/ruby/rdoc
88
- **Homepage:** https://ruby.github.io/rdoc
@@ -177,9 +177,10 @@ lib/rdoc/
177177
├── rdoc.rb # Main entry point (RDoc::RDoc class)
178178
├── version.rb # Version constant
179179
├── task.rb # Rake task integration
180-
├── parser/ # Source code parsers (Ruby, C, Markdown, RD)
180+
├── parser/ # Source code parsers (Ruby, C, RBS, Markdown, RD)
181181
│ ├── ruby.rb # Prism-based Ruby parser
182182
│ ├── c.rb # C extension parser
183+
│ ├── rbs.rb # RBS signature parser
183184
│ └── ...
184185
├── server.rb # Live-reloading preview server (rdoc --server)
185186
├── generator/ # Documentation generators
@@ -235,10 +236,16 @@ exe/
235236

236237
### Parsers and Generators
237238

238-
- **Parsers:** Prism-based Ruby (`RDoc::Parser::Ruby`), C, Markdown, RD
239+
- **Parsers:** Prism-based Ruby (`RDoc::Parser::Ruby`), C, RBS (`RDoc::Parser::RBS`), Markdown, RD
239240
- **Generators:** HTML/Aliki (default), HTML/Darkfish (deprecated), RI, POT (gettext), JSON, Markup
240241

241-
Parser tests live in the `RDocParserRubyTest` class (`test/rdoc/parser/ruby_test.rb`).
242+
Parser tests live under `test/rdoc/parser/`, including `RDocParserRubyTest` (`test/rdoc/parser/ruby_test.rb`) and `RDocParserRBSTest` (`test/rdoc/parser/rbs_test.rb`).
243+
244+
### RBS Documentation Input and Signature Merging
245+
246+
Selected `.rbs` files are first-class documentation input through `RDoc::Parser::RBS`. RBS declarations can create documentation for classes, modules, methods, attributes, and constants, or extend objects already documented from Ruby source.
247+
248+
RBS files under the project's `sig/` directory are also auto-discovered by `RDoc::RDoc` for type signature merging and live preview bookkeeping. Keep this distinction clear: `.rbs` parsing builds documentation objects, while `sig/**/*.rbs` auto-discovery feeds the existing RBS type-signature merge path.
242249

243250
### Code Object Model and Constant Aliases
244251

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,9 @@ rdoc --main README.md
4141

4242
You'll find information on the various formatting tricks you can use in comment blocks in the documentation this generates.
4343

44-
RDoc uses file extensions to determine how to process each file. File names ending `.rb` and `.rbw` are assumed to be Ruby source. Files ending `.c` are parsed as C files. All other files are assumed to contain just Markup-style markup (with or without leading `#` comment markers). If directory names are passed to RDoc, they are scanned recursively for C and Ruby source files only.
44+
RDoc uses file extensions to determine how to process each file. File names ending `.rb` and `.rbw` are assumed to be Ruby source. Files ending `.c` are parsed as C files. Files ending `.rbs` are parsed as RBS signature files. All other files are assumed to contain just Markup-style markup (with or without leading `#` comment markers). If directory names are passed to RDoc, they are scanned recursively for C, Ruby, and RBS source files.
45+
46+
RBS files can document classes, modules, methods, attributes, and constants. When RBS declarations match objects already documented from Ruby source, their comments and type signatures extend the existing documentation.
4547

4648
To generate documentation using `rake` see [RDoc::Task](https://ruby.github.io/rdoc/RDoc/Task.html).
4749

lib/rdoc/code_object/context.rb

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -313,13 +313,7 @@ def add_class(class_type, given_name, superclass = '::Object')
313313
@store.modules_hash[given_name]
314314
return enclosing if enclosing
315315
# not found: create the parent(s)
316-
names = ename.split('::')
317-
enclosing = self
318-
names.each do |n|
319-
enclosing = enclosing.classes_hash[n] ||
320-
enclosing.modules_hash[n] ||
321-
enclosing.add_module(RDoc::NormalModule, n)
322-
end
316+
enclosing = find_or_create_namespace_path ename
323317
end
324318
else
325319
name = full_name
@@ -500,11 +494,44 @@ def add_method(method)
500494
method
501495
end
502496

497+
##
498+
# Returns the owner context and local name for +constant_path+, creating
499+
# missing namespace modules. A leading +::+ resolves from the top-level.
500+
# This only resolves explicit context-tree paths; RDoc::Parser::Ruby has
501+
# parser-local lexical helpers for Ruby's nesting-dependent lookup.
502+
503+
def find_or_create_constant_owner_for_path(constant_path) # :nodoc:
504+
constant_path = constant_path.to_s
505+
owner = constant_path.start_with?('::') ? top_level : self
506+
constant_path = constant_path.delete_prefix('::')
507+
508+
owner_path, separator, name = constant_path.rpartition('::')
509+
owner = owner.find_or_create_namespace_path owner_path unless separator.empty?
510+
511+
[owner, name]
512+
end
513+
514+
##
515+
# Finds or creates the module namespace path under this context.
516+
517+
def find_or_create_namespace_path(path) # :nodoc:
518+
path.to_s.split('::').inject(self) do |owner, name|
519+
owner.classes_hash[name] ||
520+
owner.modules_hash[name] ||
521+
owner.add_module(RDoc::NormalModule, name)
522+
end
523+
end
524+
503525
##
504526
# Adds a module named +name+. If RDoc already knows +name+ is a class then
505527
# that class is returned instead. See also #add_class.
506528

507529
def add_module(class_type, name)
530+
if name.to_s.include?('::')
531+
owner, name = find_or_create_constant_owner_for_path name
532+
return owner.add_module class_type, name unless owner == self
533+
end
534+
508535
mod = @classes[name] || @modules[name]
509536
return mod if mod
510537

lib/rdoc/parser.rb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -293,5 +293,6 @@ def handle_tab_width(body)
293293
require_relative 'parser/changelog'
294294
require_relative 'parser/markdown'
295295
require_relative 'parser/rd'
296+
require_relative 'parser/rbs'
296297
require_relative 'parser/ruby'
297298
require_relative 'parser/ruby_colorizer'

lib/rdoc/parser/rbs.rb

Lines changed: 275 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,275 @@
1+
# frozen_string_literal: true
2+
3+
require 'rbs'
4+
5+
##
6+
# Parse RBS signature files as first-class RDoc input.
7+
8+
class RDoc::Parser::RBS < RDoc::Parser
9+
RBS_FILE_EXTENSION = /\.rbs$/
10+
11+
parse_files_matching RBS_FILE_EXTENSION
12+
13+
def scan
14+
_, _, decls = ::RBS::Parser.parse_signature(@content)
15+
decls.each do |decl|
16+
parse_decl decl, @top_level
17+
end
18+
@top_level
19+
end
20+
21+
private
22+
23+
def record_object_location(object, location)
24+
object.line = location.start_line if location
25+
26+
if RDoc::ClassModule === object
27+
@top_level.add_to_classes_or_modules object unless
28+
@top_level.classes_or_modules.include? object
29+
end
30+
31+
object.record_location @top_level
32+
object
33+
end
34+
35+
def rdoc_comment_for(decl)
36+
rbs_comment = decl.comment if decl.respond_to?(:comment)
37+
return unless rbs_comment
38+
39+
# TODO: Run RBS comments through RDoc's directive preprocessor so
40+
# directives like :nodoc: affect the documented object.
41+
comment = RDoc::Comment.new rbs_comment.string, @top_level
42+
comment.format = 'markdown'
43+
comment
44+
end
45+
46+
def local_module_name(type_name, context)
47+
name = type_name.to_s
48+
return name if name.start_with?('::')
49+
50+
namespace_names = context == @top_level ? [] : context.full_name.split('::')
51+
52+
namespace_names.length.downto(1) do |length|
53+
qualified_name = namespace_names.take(length).join('::')
54+
if module_name = @top_level.find_module_named("#{qualified_name}::#{name}")
55+
return module_name.full_name
56+
end
57+
end
58+
59+
name
60+
end
61+
62+
def merge_documentation(object, comment, type_signature_lines)
63+
if comment
64+
object.comment = if object.comment.empty?
65+
comment
66+
else
67+
merge_comments object, comment
68+
end
69+
end
70+
71+
# TODO: Track RBS-owned documentation overlays so incremental reparsing can
72+
# replace stale comments and signatures from the previous RBS parse.
73+
object.type_signature_lines ||= type_signature_lines
74+
end
75+
76+
def merge_comments(object, comment)
77+
document = RDoc::Markup::Document.new
78+
document.concat object.parse(object.comment).parts
79+
document << RDoc::Markup::Rule.new(1)
80+
document.concat comment.parse.parts
81+
82+
# Keep this text separator in sync with the Rule node above.
83+
merged_comment = RDoc::Comment.new "#{object.comment}\n---\n#{comment}", comment.location
84+
merged_comment.format = 'markdown'
85+
merged_comment.document = document
86+
merged_comment
87+
end
88+
89+
def attr_rw_matches?(existing_rw, new_rw)
90+
existing_rw.each_char.any? { |rw| new_rw.include? rw }
91+
end
92+
93+
def merge_attribute_methods(context, name, rw, singleton, comment, type_signature_lines)
94+
method_names = []
95+
method_names << name if rw.include?('R')
96+
method_names << "#{name}=" if rw.include?('W')
97+
98+
methods = method_names.map { |method_name| context.find_method(method_name, singleton) }
99+
methods.compact.each do |method|
100+
merge_documentation method, comment, type_signature_lines
101+
end
102+
103+
methods.any?
104+
end
105+
106+
def rdoc_method_name(decl)
107+
rbs_constructor_decl?(decl) ? 'new' : decl.name.to_s
108+
end
109+
110+
def rdoc_method_singleton?(decl)
111+
# TODO: RBS `self?` methods are :singleton_instance and should add both a
112+
# singleton method and a private instance method.
113+
rbs_constructor_decl?(decl) || decl.singleton?
114+
end
115+
116+
def rdoc_method_visibility(decl)
117+
rbs_constructor_decl?(decl) ? :public : decl.visibility
118+
end
119+
120+
def rbs_constructor_decl?(decl)
121+
decl.kind == :instance && decl.name == :initialize
122+
end
123+
124+
def parse_attr_decl(decl, context)
125+
rw = case decl
126+
when ::RBS::AST::Members::AttrReader
127+
'R'
128+
when ::RBS::AST::Members::AttrWriter
129+
'W'
130+
when ::RBS::AST::Members::AttrAccessor
131+
'RW'
132+
end
133+
134+
comment = rdoc_comment_for decl
135+
type_signature_lines = [decl.type.to_s]
136+
name = decl.name.to_s
137+
singleton = decl.kind == :singleton
138+
if attribute = context.find_attribute(name, singleton)
139+
merge_documentation attribute, comment, type_signature_lines if
140+
attr_rw_matches? attribute.rw, rw
141+
return
142+
end
143+
144+
if merge_attribute_methods(context, name, rw, singleton, comment, type_signature_lines)
145+
return
146+
end
147+
148+
attribute = RDoc::Attr.new(
149+
name,
150+
rw,
151+
comment,
152+
singleton: singleton
153+
)
154+
record_object_location attribute, decl.location
155+
attribute.type_signature_lines = type_signature_lines
156+
context.add_attribute attribute
157+
attribute.visibility = decl.visibility if decl.visibility
158+
end
159+
160+
def parse_class_decl(decl, context)
161+
owner, name = context.find_or_create_constant_owner_for_path decl.name
162+
superclass = decl.super_class&.name&.to_s || '::Object'
163+
klass = owner.add_class RDoc::NormalClass, name, superclass
164+
record_object_location klass, decl.location
165+
comment = rdoc_comment_for decl
166+
klass.add_comment comment, @top_level if comment
167+
168+
decl.members.each { |member| parse_decl member, klass }
169+
end
170+
171+
def parse_constant_decl(decl, context)
172+
constant = RDoc::Constant.new decl.name.to_s, decl.type.to_s,
173+
rdoc_comment_for(decl)
174+
record_object_location constant, decl.location
175+
context.add_constant constant
176+
end
177+
178+
def parse_decl(decl, context)
179+
case decl
180+
when ::RBS::AST::Declarations::Class
181+
parse_class_decl decl, context
182+
when ::RBS::AST::Declarations::Module, ::RBS::AST::Declarations::Interface
183+
parse_module_decl decl, context
184+
when ::RBS::AST::Declarations::ClassAlias,
185+
::RBS::AST::Declarations::ModuleAlias
186+
# TODO: Add RBS class and module aliases to the RDoc store.
187+
nil
188+
else
189+
parse_member_decl decl, context
190+
end
191+
end
192+
193+
def parse_extend_decl(decl, context)
194+
extend_decl = RDoc::Extend.new local_module_name(decl.name, context),
195+
rdoc_comment_for(decl)
196+
record_object_location extend_decl, decl.location
197+
context.add_extend extend_decl
198+
end
199+
200+
def parse_include_decl(decl, context)
201+
include_decl = RDoc::Include.new local_module_name(decl.name, context),
202+
rdoc_comment_for(decl)
203+
record_object_location include_decl, decl.location
204+
context.add_include include_decl
205+
end
206+
207+
def parse_member_decl(decl, context)
208+
case decl
209+
when ::RBS::AST::Declarations::Constant
210+
parse_constant_decl decl, context
211+
when ::RBS::AST::Members::MethodDefinition
212+
parse_method_decl decl, context
213+
when ::RBS::AST::Members::Alias
214+
parse_method_alias_decl decl, context
215+
when ::RBS::AST::Members::AttrReader,
216+
::RBS::AST::Members::AttrWriter,
217+
::RBS::AST::Members::AttrAccessor
218+
parse_attr_decl decl, context
219+
when ::RBS::AST::Members::Include
220+
parse_include_decl decl, context
221+
when ::RBS::AST::Members::Extend
222+
parse_extend_decl decl, context
223+
when ::RBS::AST::Members::Private,
224+
::RBS::AST::Members::Public
225+
# TODO: Track standalone RBS visibility members.
226+
nil
227+
end
228+
end
229+
230+
def parse_method_alias_decl(decl, context)
231+
alias_def = RDoc::Alias.new(
232+
decl.old_name.to_s,
233+
decl.new_name.to_s,
234+
rdoc_comment_for(decl),
235+
singleton: decl.kind == :singleton
236+
)
237+
record_object_location alias_def, decl.location
238+
context.add_alias alias_def
239+
end
240+
241+
def parse_method_decl(decl, context)
242+
comment = rdoc_comment_for decl
243+
type_signature_lines = decl.overloads.map { |overload| overload.method_type.to_s }
244+
method_name = rdoc_method_name(decl)
245+
singleton = rdoc_method_singleton?(decl)
246+
visibility = rdoc_method_visibility(decl)
247+
248+
if method = context.find_method(method_name, singleton)
249+
merge_documentation method, comment, type_signature_lines
250+
return
251+
end
252+
253+
method = RDoc::AnyMethod.new method_name, singleton: singleton
254+
record_object_location method, decl.location
255+
method.type_signature_lines = type_signature_lines
256+
257+
if loc = decl.location
258+
method.start_collecting_tokens :ruby
259+
method.add_token line_no: loc.start_line, char_no: 1, text: loc.source
260+
end
261+
262+
method.comment = comment if comment
263+
context.add_method method
264+
method.visibility = visibility if visibility
265+
end
266+
267+
def parse_module_decl(decl, context)
268+
mod = context.add_module RDoc::NormalModule, decl.name.to_s
269+
record_object_location mod, decl.location
270+
comment = rdoc_comment_for decl
271+
mod.add_comment comment, @top_level if comment
272+
273+
decl.members.each { |member| parse_decl member, mod }
274+
end
275+
end

0 commit comments

Comments
 (0)