nREPL network REPL server (WIP)

[jasalt]

Disliking that async AMPHP server code needs a restart after every code change so I started to try puzzle together nrepl server for Phel using amphp libraries (byte-stream) that could run in same event loop as the http-server. Then application could be developed interactively.

In general, I think it could provide better text editor integrations for interactive development with existing tooling (Emacs Cider, VSCode Calva, Cursive, Neovim ...), if it can be made work.

Wrapping my head around bytestreams currently. Noted some useful pieces in ilt-phel for inspecting Phel runtime info.

[jasalt]

Basic "http repl" example working so far, just to see that code can modify the server process runtime state before doing any more advanced plumbing beforehand. Based on AMPHP event-source.php example, using Datastar library for swapping streamed html partials to view.

rend.mp4

Source: https://codeberg.org/jasalt/phel-snake-online/src/branch/main/src/httprepl.phel

[jasalt]

nREPL allows using various transits but default is bencode, so wrapped a encoding/decoding PHP library so that it can be installed as composer dependency and called with Phel functions / data structures: https://packagist.org/packages/jasalt/phel-bencode

Streaming encoding/decoding is not implemented which seems to be the main way in mainstream nRepl server implementations but buffering line by line as done currently seems to work good enough for now while testing the overall implementation.

[jasalt]

Early version with :describe and :eval nREPL operations (https://gitlab.com/jasalt/phel-nrepl) e.g.:

$ composer install
$ vendor/bin/phel run src/nrepl.phel

# on other terminal

$ nc localhost 8888
nREPL server started on port 8888 on host 127.0.0.1 - nrepl://127.0.0.1:8888
d2:op8:describee
d3:opsl4:eval5:clone8:describee6:statusl4:donee8:versionsd4:phel8:dev-main3:php6:8.2.26ee
d4:code7:(+ 1 1)2:op4:evale
d6:statusl4:donee5:valuei2ee

Talking bencoded nREPL commands over socket to nREPL process, raw messages being:

d2:op8:describee = {:op "describe}

• (Phel handler-fn)

d4:code7:(+ 1 1)2:op4:evale = {:op "eval" :code "(+ 1 1)"}

• (Phel handler-fn)

[jasalt]

I got the Calva/VSCode connection working! There's some nREPL client specific glitches still, but I think connection should work with other text editor clients also (Neovim, Cursive, ...). Working primarily with Emacs Cider client myself but this should allow for better collaboration across editors regarding REPL stuff.

linux_desktop_recording-2025-03-30_13.48.35.mp4

Mirrored the repo to this platform also for convenience https://github.com/jasalt/phel-nrepl where issues can be posted with reference to Phel issue tracker.

Continuing with tidying up the ops so completions would render properly etc. and looking how to make get REPL require working from non standard Phel Repl context (related Phel issue #766)...

[jasalt]

Everything still very experimental but I updated the https://github.com/jasalt/phel-nrepl project to use latest Phel version, cleaned up the codebase and did improvements in:

• Up to date nREPL session examples with findings in https://github.com/jasalt/phel-nrepl/tree/master/examples
• Namespace loading (ns form evaluation with requires works)
• Asynchronic eval (e.g. initiating amphp http server on another socket via nrepl doesn't block the nrepl server itself)

The clj-nrepl-eval CLI tool is also somewhat working (screenshot), which can help connecting live PHP/Phel process REPL with LLM coding tools e.t.c.

Developing/testing with Emacs Cider, Calva support may have regressed since adding initial support but checking it a bit later.

[jasalt]

Most head scratching has been in getting the namespaces and requires working when evaluating ns forms. Currently it seems to mostly work but I don't think it's very stable / complete yet.

E.g. one confusing thing is that *ns* does not update consistently after namespace change by ns form evaluation but (php/-> (php/:: \Phel\Compiler\Infrastructure\GlobalEnvironmentSingleton (getInstance)) (getNs)) result is updated correctly and symbol resolution within namespace(s) is seemingly working.

I'd appreciate if @Chemaclass and/or others (incl. @SauronBot, haha) have time to review if this could be made in more stable / sustainable way, or if Phel core could be improved in way for supporting something like this more natively.

For getting custom REPL environment like this working (initialized from within a phel run process) I dug through the vanilla REPL sources and found these being essential so far:

Setting REPL_MODE constant for keeping require available

(php/-> (php/:: \Phel\Lang\Registry (getInstance))
          (addDefinition
           (php/:: \Phel\Shared\CompilerConstants PHEL_CORE_NAMESPACE)
           (php/:: \Phel\Shared\ReplConstants REPL_MODE)
           true))

In https://github.com/jasalt/phel-nrepl/blob/2a3b0afaaa106a2e6d707ab71f5fa7d1afa1749f/src/nrepl.phel#L258-L262 from:

phel-lang/src/php/Run/Infrastructure/Command/ReplCommand.php

Line 112 in c58dab5

Phel::addDefinition(CompilerConstants::PHEL_CORE_NAMESPACE, ReplConstants::REPL_MODE, true);

NamespaceLoader.php loadPhelNamespaces()

(php/-> (php/new \Phel\Run\RunFacade) (loadPhelNamespaces))

In https://github.com/jasalt/phel-nrepl/blob/2a3b0afaaa106a2e6d707ab71f5fa7d1afa1749f/src/nrepl.phel#L270 from:

phel-lang/src/php/Run/Application/NamespaceLoader.php

Lines 31 to 72 in c58dab5

	public function loadPhelNamespaces(?string $replStartupFile = null): void
	{
	if ($replStartupFile === null) {
	$replStartupFile = $this->defaultReplStartupFile;
	}
	if (!file_exists($replStartupFile)) {
	return;
	}
	$namespace = $this->buildFacade
	->getNamespaceFromFile($replStartupFile)
	->getNamespace();
	$srcDirectories = [
	dirname($replStartupFile),
	...$this->commandFacade->getAllPhelDirectories(),
	];
	$namespaceInformation = $this->buildFacade->getDependenciesForNamespace(
	$srcDirectories,
	[$namespace, 'phel\\core'],
	);
	foreach ($namespaceInformation as $info) {
	$file = $info->getFile();
	if (!isset(self::$loadedFiles[$file])) {
	$this->buildFacade->evalFile($file);
	self::$loadedFiles[$file] = true;
	}
	}
	Phel::addDefinition(CompilerConstants::PHEL_CORE_NAMESPACE, '*file*', '');
	$cwd = getcwd();
	if ($cwd !== false) {
	$srcDirectories[] = $cwd;
	}
	// Hack: Set source directories for the repl
	Phel::addDefinition('phel\\repl', 'src-dirs', $srcDirectories);
	}

Most confusing part. Without this all Phel core namespaces that weren't required in the nrepl.phel ns form initially were missing their definitions after requiring them via ns form evaluation, while the refers were getting updated.

Also some code smell with "Hack" mentioned is persisting :):

phel-lang/src/php/Run/Application/NamespaceLoader.php

Line 70 in c58dab5

// Hack: Set source directories for the repl

[Chemaclass]

Great progress @jasalt! We've landed a full batch of PRs to help with the core-side blockers (all merged to main):

Foundation:

• fix(repl): fix ns form handling and *ns* hyphen munging #1120
• feat(repl): REPL tooling improvements for nREPL support #1121

New since last update:

• feat(repl): capture stdout during eval in EvalResult #1123
• feat(repl): add load-file function for evaluating source files in REPL #1124
• feat(repl): add symbol-info for structured metadata lookup #1125
• fix: return "false" and "true" from str for boolean values #1126
• feat(compiler): store parameter names as arglists in function metadata #1127
• feat(repl): add source function to retrieve definition source code #1128
• feat(repl): rollback GlobalEnvironment state on eval errors #1129
• feat(repl): add structured stack frames to EvalError #1130
• feat(repl): add ns-publics, ns-aliases, ns-refers for namespace introspection #1131
• feat(repl): add find-fn for structured function search #1132
• feat(repl): add test-ns function for running namespace tests interactively #1133

These should cover the main nREPL ops: eval, info, complete, load-file, lookup/source, stacktrace, and test. Would love to hear which of these help and what else is needed!

[jasalt]

Thank's a ton @Chemaclass, the changes must reduce days of work to get this in working condition.

While I'm poking with this currently using Emacs Cider let me know also if you see something I could do to make this working with your tooling e.t.c.

Looking to have better testing setup also in place of current "notebook" style documentation while it's still in a quite exploratory state.

Feedback, forks, PR's and other contributions to the project are welcome.

[jasalt]

I'm not sure yet about best way for initializing the environment but for testing with the latest changes I took the call to (php/-> (php/new \Phel\Run\RunFacade) (loadPhelNamespaces)) out from here:

jasalt/phel-nrepl@8a2801e#diff-5776e1b23e2507a9c69951dc48632e06fbbcf1faa063f848974b4f785e383653L334-L335

With that it seems that the ns form evaluation works with :require's (due to the ns form handling issue fix, example nrepl session), but it seems that require from phel\repl maybe doesn't use the same fallback for resolving the definitions and fails still if the (loadPhelNamespaces) has not been called beforehand.

Example nRepl session using require here: https://github.com/jasalt/phel-nrepl/blob/master/examples/require-demo.phel

Question @Chemaclass: is this a defect or intended behavior?

Thinking if I should revert removing the (loadPhelNamespaces) for now or wait that require "just worked" in this scenario.

[jasalt]

Best minimal reproduction I could make so far (Phel v0.30.0-beta#fa207dd / PHP 8.4), save as demo.phel and phel run demo.phel:

(ns require-demo-repro
  (:require phel\repl))

(repl/eval-str "(ns demo-foo
                  (:require phel\\repl :refer [require]))
                ;; (php/-> (php/new \Phel\Run\RunFacade) (loadPhelNamespaces))  ; uncomment to see require working
                (require phel\\html)
                (print (type html/raw-string))")

Result:

$ phel run demo.phel
[PHEL001] Cannot resolve symbol 'html/raw-string'. Did you mean 'read-string'?
in string:5

5| (print (type html/raw-string))
                               ^^^^^^^^^^^^^^^

With uncommented line having call to (loadPhelNamespaces):

phel run demo.phel
:function:function

Shows that this way it works. There's duplicate :function printed due to output issue but I don't think it's related to this.

[jasalt]

Confirmed #1142 solved the issue, and also the above require-demo-repro works without uncommenting the (loadPhelNamespaces). Great!

[Chemaclass]

@jasalt, all three issues from your last comments are fixed in #1142 (just merged):

• Require without loadPhelNamespaces(): this was a defect. The runtime require macro now lazily resolves source directories via CommandFacade when src-dirs hasn't been initialized. Your repro now works without the workaround:

  $ phel run demo.phel
  :function

• You can remove the (loadPhelNamespaces) call from phel-nrepl startup if you want; require and eval-strwith(:require ...)` both work standalone now.

• Duplicate output (phel run command duplicates output on first run #1139): root cause was FileEvaluator executing code twice on cache miss: once during compileForCache (needed for macros) and again via require on the cached file. Fixed by removing the redundant require.

• ns consistency: verified this is already working correctly after the earlier PRs. Both *ns* and GlobalEnvironment::getNs() stay in sync after eval-str evaluates ns forms. Added regression tests to keep it that way.

Let me know how these work with phel-nrepl, happy to look at anything else that comes up.

[jasalt]

Have been spending some time testing and getting eval working in more scenarios and finally got Amphp event source HTTP server demo working for a nice REPL live web coding example.

Kooha-2026-04-07-17-54-00.mp4

Example sources: https://github.com/jasalt/phel-nrepl/blob/b0d49dfc882011efd2c47d8a59bd5616c22c54a8/examples/event-source.phel

[jasalt]

Found out via Mastodon that @leobm has been working on Phel nREPL project too https://github.com/leobm/phel-nrepl with more focus on VSCode Calva and based on ReactPHP though he was interested to switch to Amphp also. We can possibly collaborate on the project to get it in working condition faster in way or another.

[jasalt]

@leobm there's quite many improvements recently on Phel side that make the implementation much more straightforward. Above message lists many: #794 (comment). I've implemented some of them in my version so far and left issues about rest.

Also there's namespace aliasing so e.g. clojure.stacktrace matches phel\stacktrace and you could write such Phel namespace that provides the functionality that Calva expects if it calls for clojure ns functions automatically (Cider does it at times) #1210 & #1251 ... No need to check clojure-specific?, in some scenarios at least.

[Chemaclass]

Update: native nREPL server shipped in main (#1507)

Big thanks to @jasalt and @leobm — your external implementations and the feedback loop on #1120-#1142, #1258/#1265 made landing this straightforward.

./bin/phel nrepl now starts a bencode-over-TCP server (default 127.0.0.1:7888). Supported ops:

Op	Purpose
eval	evaluate code, stream stdout/stderr/value
clone / close	session lifecycle
describe	capability discovery
load-file	slurp-and-eval a file
interrupt	stop running eval
completions	prefix completions with type info
lookup / info / eldoc	symbol metadata (arglists, doc, file:line)

Accept loop uses PHP Fibers (one per client); bencode codec is a standalone lib. Tested against CIDER and Calva. Docs: docs/nrepl-guide.md.

Would love feedback on:

• Ops your clients need that aren't implemented (streaming chunked output, eval variants, ls-sessions, stdin, ...)
• Whether jasalt/phel-nrepl and leobm/phel-nrepl should converge onto the core server, or stay as testbeds for ideas that haven't landed yet
• Any regressions vs. the external implementations

Issues welcome for anything missing.