Skip to content

Release ctrld-client - #266

Open
cuonglm wants to merge 277 commits into
mainfrom
release-branch-v2.0.0
Open

cuonglm wants to merge 277 commits into
mainfrom
release-branch-v2.0.0

Conversation

@cuonglm

@cuonglm cuonglm commented Oct 9, 2025 •

Copy link
Copy Markdown
Collaborator

Major Release

This is the first release of ctrld-client.

Added

  • Rule Matching Engine: Implemented new modular rule matching engine infrastructure with support for configurable rule evaluation order (infrastructure ready, not yet exposed to users)
  • Modular CLI Architecture: Split monolithic CLI command structure into focused, maintainable command files
  • Context Support: Added context.Context support throughout configuration methods for better cancellation and timeout handling

Improvements

  • Logging System: Migrated from zerolog to uber zap logging package with improved performance, better structured logging, and enhanced extensibility
  • CLI Architecture: Refactored monolithic commands.go (1,397 lines) into 13 focused command files, improving maintainability and testability
  • DNS Proxy: Major refactoring of DNS proxy implementation with better code organization, improved error handling, and enhanced separation of concerns
  • Client Information System: Enhanced client discovery and information management with improved DHCP, mDNS, ARP, NDP, and hosts file parsing
  • Service Management: Improved service lifecycle management, enhanced reload functionality, and better status reporting across all platforms
  • Network/OS Abstractions: Improved network and OS abstraction layers with reduced code duplication and more consistent behavior across platforms
  • Configuration System: Added context support and improved logging throughout configuration initialization and bootstrap operations
  • Code Organization: Removed ~3,000 lines of deprecated router-specific code, focusing development on core DNS proxy functionality

Fixes

  • Improved error handling and recovery mechanisms throughout the codebase
  • Enhanced service state management and lifecycle handling

Breaking Changes

⚠️ Server and Router-Specific Integrations Removed: All server platforms and router-specific integration code has been removed, including support for:

  • DD-WRT, dnsmasq, EdgeOS, Firewalla, AsusWRT-Merlin, Netgear/Orbi/Voxel, OpenWRT, Synology, Tomato, and Ubiquiti UniFi OS routers
  • Windows Server

If you were using ctrld with any of these router platforms, you will need to use alternative deployment methods. See the migration guide for details.

Note: All other functionality remains backward compatible. Existing configuration files and CLI commands continue to work without changes.

@cuonglm
cuonglm force-pushed the release-branch-v2.0.0 branch 2 times, most recently from f38c9ae to 90eddb8 Compare October 9, 2025 13:51
@cuonglm
cuonglm force-pushed the release-branch-v2.0.0 branch 2 times, most recently from 1c74fc4 to f9d0263 Compare November 12, 2025 08:21
@cuonglm cuonglm changed the title [WIP] Release branch v2.0.0 Release branch v2.0.0 Nov 12, 2025
@cuonglm
cuonglm force-pushed the release-branch-v2.0.0 branch 4 times, most recently from 0d4b697 to d0e66b8 Compare December 17, 2025 08:28
@cuonglm
cuonglm force-pushed the release-branch-v2.0.0 branch 4 times, most recently from de415df to 1fbbb14 Compare March 10, 2026 10:42
cuonglm added 17 commits April 30, 2026 19:19
Add CommandRunner interface and ServiceManager types to support
dependency injection and better separation of concerns in command handling.
Create separate file for log command handling to improve code organization.
Add LogCommand struct with SendLogs and ViewLogs methods to handle
log-related operations with proper error handling and dependency injection.
Create separate file for service command handling to improve code organization.
Add ServiceCommand struct with Install, Uninstall, Start, Stop, and Status
methods to handle service operations with proper error handling and dependency
injection.
… logic

Create separate file for upgrade command handling to improve code organization.
Add UpgradeCommand struct with Upgrade method that includes all original logic:
channel management, service restart, rollback handling, and version verification.
Includes InitUpgradeCmd function with proper argument validation and privilege checks.
… logic

Create separate file for clients command handling to improve code organization.
Add ClientsCommand struct with ListClients method that includes all original logic:
service status checks, HTTP requests, source mapping, metrics handling, and table
formatting. Includes InitClientsCmd function that creates proper command hierarchy
with clients parent command and list sub-command.
Create separate file for interfaces command handling to improve code organization.
Add InterfacesCommand struct with ListInterfaces method that handles the
logic to list current system interfaces.
Port all special logic from original alias commands:
- startCmdAlias: custom Args validation, startOnly logic, iface handling
- stopCmdAlias: iface flag handling and argument passing
- restartCmdAlias: simple delegation to restartCmd.RunE
- reloadCmdAlias: simple delegation to reloadCmd.RunE
- statusCmdAlias: simple delegation to statusCmd.RunE
- uninstallCmdAlias: iface flag handling and argument passing

All aliases now have exact same behavior as original implementation
including proper flag inheritance and argument handling.
Remove the old initLogCmd function from commands.go and update cli.go
to use the new InitLogCmd function from commands_log.go. Complete
the log command refactoring by adding the missing InitLogCmd function
with proper command structure and error handling.
…mplete logic

Replace individual service command initialization with unified InitServiceCmd()
that creates a complete service command hierarchy. Port all original logic
from initStartCmd, initStopCmd, initRestartCmd, initReloadCmd, initStatusCmd,
and initUninstallCmd into ServiceCommand methods with proper dependency injection.

Key changes:
- Port complete Start logic including config validation, service installation,
  DNS management, and self-check functionality
- Port complete Stop logic with deactivation pin validation and DNS cleanup
- Port complete Restart logic with config validation and DNS restoration
- Port complete Reload logic with HTTP status handling and restart fallback
- Port complete Status logic with proper exit codes
- Port complete Uninstall logic with cleanup file removal
- Add all necessary flags to service commands (iface, pin, etc.)
- Use InitInterfacesCmd() for interfaces subcommand
- Simplify cli.go by replacing multiple init calls with single InitServiceCmd()

This refactoring eliminates code duplication, improves maintainability, and
ensures all service commands have their complete original functionality.
Remove all unused service command functions (initStartCmd, initStopCmd,
initRestartCmd, initReloadCmd, initStatusCmd, initUninstallCmd,
initInterfacesCmd, initClientsCmd, initUpgradeCmd, initServicesCmd)
from commands.go since they have been replaced by modular implementations
in dedicated files. Keep only essential functions: CommandRunner interface,
ServiceManager struct, NewServiceManager function, Status method,
initRunCmd function, and filterEmptyStrings function.

Update cli.go to use InitClientsCmd() and InitUpgradeCmd() instead of
the old init functions. Clean up unused imports and simplify
filterEmptyStrings implementation.

This reduces commands.go from 1202 lines to 103 lines (91% reduction)
and eliminates code duplication while improving maintainability.
Create commands_run.go following the same modular pattern as other
command files. Move initRunCmd logic to InitRunCmd function with
consistent naming and complete functionality preservation.

Update cli.go to use InitRunCmd() instead of initRunCmd() and clean
up commands.go by removing the old function and unused imports.

This completes the modular refactoring pattern where each command type
has its own dedicated file with focused responsibility.
…nterface

Rename service_manager.go to commands_service_manager.go to follow the
established naming pattern with other command files.

Remove the unused CommandRunner interface from commands.go since it's not
being used anywhere in the codebase. Clean up unused imports.

This improves consistency in file naming and removes dead code.
…nstall

Add missing selfDeleteExe() call and supportedSelfDelete check that were
present in the original initUninstallCmd function. This ensures the
uninstall command properly handles self-deletion of the binary when
cleanup is enabled.

The original logic included:
- selfDeleteExe() call for self-deletion
- supportedSelfDelete check for platform-specific behavior
- Proper error handling and logging

This completes the porting of all functionality from the original
initUninstallCmd to the new ServiceCommand.Uninstall method.
- Move ServiceCommand.Start to commands_service_start.go
- Move ServiceCommand.Stop to commands_service_stop.go
- Move ServiceCommand.Restart to commands_service_restart.go
- Move ServiceCommand.Reload to commands_service_reload.go
- Move ServiceCommand.Status to commands_service_status.go
- Move ServiceCommand.Uninstall to commands_service_uninstall.go
- Move createStartCommands to commands_service_start.go
- Clean up imports in commands_service.go
- Remove all method implementations from main service file

This refactoring improves code organization by:
- Separating concerns into focused files
- Making navigation easier for developers
- Reducing merge conflicts between different commands
- Following consistent modular patterns
- Reducing commands_service.go from ~650 lines to ~50 lines

Each method is now co-located with its related functionality,
making the codebase more maintainable and easier to understand.
…nciple

Remove rootCmd.AddCommand call from createStartCommands function.
The function should only create and return commands, not add them
to the root command hierarchy. This responsibility belongs to the
caller (InitServiceCmd).

This change improves:
- Separation of concerns: function has single responsibility
- Testability: no hidden side effects
- Flexibility: caller controls command registration
- Clean architecture: follows principle of no hidden dependencies
To keep the same order with v1.0 service sub-commands list.
- Split initializeServiceManager into two methods:
  * initializeServiceManager(): Simple method using default configuration
  * initializeServiceManagerWithServiceConfig(): Advanced method for custom config
- Simplify NewServiceCommand() to return *ServiceCommand without error
- Update all service command methods to use appropriate initialization:
  * Start: Uses initializeServiceManagerWithServiceConfig() for custom args
  * Stop/Restart/Reload/Status/Uninstall: Use simple initializeServiceManager()
- Remove direct access to sc.serviceManager.svc/prog in favor of lazy initialization
- Improve separation of concerns and reduce code duplication
Dev Scribe and others added 30 commits November 16, 2026 15:29
…er build

"ctrld upgrade" takes --version to install a specific release, or
--branch to install the latest build of a branch. The downloaded binary
must report the requested version before it is installed.

The new "ctrld rollback" command installs an older build the same way:
the release before the installed one by default, or the one given by
--version or --branch.
…nt calls

master adaptation: chassisValues takes master's per-call *Logger (master
has no ProxyLogger) and keeps master's debug line for a failed lookup. The
mobile test stubs master's root-package isIOS hook, which is what makes
systemMetadata skip host lookups on master.
On a network that gives the Mac a public IPv4 address with no NAT, pf
redirects the Mac's own lookups to the 127.0.0.1 listener and keeps their
public source address. The WAN-client check then refused every one of them
(REFUSED), and refused the watchdog's pf probe too, which turned into a pf
reload about every 30 s.

The existing exemption covered only a [::1] listener, for the same reason:
queries on it always come from this machine. Other hosts cannot send packets
to a loopback address, so widen the exemption to any loopback listener
(127.0.0.0/8 and [::1]). Listeners on LAN, public or wildcard addresses
still refuse WAN sources unless allow_wan_clients is set.
The DNS handler read p.cfg.Listener with no lock, while a reload writes
p.cfg under p.mu. Under -race, TestReloadThatDropsARunningListener found
it: the read of the handler raced with the write of runWait, and with the
decode of the reload config into the same map. The handler now reads the
config of its listener under p.mu.

TestDeviceCheckCountStartsAgainAfterTheCoolOff set the package variable
deviceCheckCoolOff, while the cool-off goroutine of an earlier device
check test could still read it. The cool-off is now a field of the prog
that a test sets before the run, and deviceCheckCoolOff is a constant.
On macOS in Intercept mode, a fallback-mode Internal Domain resolver
that is reachable only through a VPN is marked down while the VPN is
off. proxy() then skips it and goes straight to the network fallback,
and only an answer to a background re-check marks it up again. That
re-check runs at most every 30 seconds and only after a lookup has
already skipped the resolver, so nothing brings the resolver back when
the VPN connects. A field log showed resolvers staying down for hours
to days across dozens of reconnects.

The first Internal Domain lookups after the VPN came back therefore got
the LAN resolver's NXDOMAIN for a VPN-only name, and macOS cached it.

refreshDNSAfterVPNSettle now marks every down fallback-mode Internal
Domain resolver up, so the next lookup asks it. Explicit-only resolvers
are never skipped and keep their state, as do the Control D upstream
and the OS resolver. A resolver that is still unreachable costs the
next lookup one timeout before the monitor marks it down again, so the
off-network shortcut from b5d7fec3 still applies after that.
The build stage ran on the target platform, so a multi-platform build
ran the Go toolchain under qemu. On hosts with an older qemu-user
(Ubuntu 22.04 ships 6.2) Go crashed building linux/arm/v7, and
scripts/build.sh did not fail on a build error. The artifact glob could
then fail late, or match a ctrld-client binary copied in from the host
instead of the one this build produced.

In both docker/Dockerfile and docker/Dockerfile.debug, run the build
stage on BUILDPLATFORM and cross-compile for the target with
scripts/build.sh linux/amd64, linux/arm64 or linux/armv7. Then move
the binary built for that target, by its exact name, to a fixed path,
and fail when it is missing. scripts/build.sh now exits non-zero when
go generate or go build fails, and .dockerignore keeps ctrld-* files
from the host out of the build context, so a leftover binary can never
stand in for a failed build. Drop the outdated CTRLD_NO_QF variable.
Go 1.27.4 and 1.26.11 fix 12 security issues in the standard library
and toolchain (CVE-2026-97033 to CVE-2026-97043, CVE-2026-97054).
Every ctrld build on go1.26.10 / go1.27.3 is affected.

- .gitlab-ci.yml: golang:1.26.10-bookworm -> golang:1.27.4-bookworm.
  CI no longer needs to hold back on 1.26.x: staticcheck 2026.2.2 reads
  go1.27 export data (dominikh/go-tools#1832, closed). Drop the TODO.
- .gitlab-ci.yml: staticcheck 2026.2 -> 2026.2.2. 2026.2 still fails on
  go1.27.4 with "export data version 5 is greater than maximum
  supported version 4".
- go.mod: toolchain go1.26.10 -> go1.27.4. The go 1.26.0 directive is
  unchanged, so GODEBUG defaults stay at their 1.26 values.
- docker/Dockerfile, docker/Dockerfile.debug: golang:1.27.3-bookworm ->
  golang:1.27.4-bookworm.
master adaptation: master logs through zap, and its initConsoleLogging
and switchLogPath already build a new logger (zap.New, zap.NewNop) with
no inherited fields, so a line gets one time field after any number of
starts. The main.go, log_writer.go and log_redact_source_test.go hunks,
the one-time-field case and Test_switchLogPath_keepsOneTimeField are left
out. Master's tryUpdateListenerConfig takes no logger, so the default
config path only adds setListenerDefaultValue. The service-reset and
file-carry cases check logFileLevel, the level of master's log files,
in place of zerolog's global level. The cd-listener-bind case and
consoleOutput are not on master and stay out; master's
start-nameservers-logged case stays in the mode list.
master adaptation: the listener key goes through master's handleDNSQuery,
which reads listenerConfig after the semaphore (race fix 10f4e1ed) and
passes the key to processStandardQuery as listenerNum, in place of
v1.0's inline serveDNS handler. keepMobileListener takes master's
*ctrld.Logger, and the new log lines are capitalized ("Reload: the DNS
server ..."). Master's applyFetchedResolverConfig already warns with
cfgErr, so on that path only the call goes through the new
updateCustomLastFailed hook; the restart validation in cli.go gets the
cfgErr fix. The tests use master's context-taking
monitorNetworkChangesFn, store the prog logger, read the run's log_path
file for the reload lines, use discardLogger in place of zerolog.Nop, and
match master's capitalized messages.
master adaptation: the code and tests apply as on v1.0. In the doc,
master's `http_status` paragraph said a non-JSON answer has no status and
is reported as API_UNREACHABLE; the v1.0 paragraph replaces it. The code
line after it keeps master's wording ("API_REJECTED or API_DEVICE_INVALID
for a permanent rejection"), because that line is not part of this
change.
master adaptation: master's recovery is split, so the device DNS guards go
into completeRecoveryWork's traditional flow (the reset before the
nameserver read and the setDNS after it), and the recovered line sits in
handleRecoveryForTransition with master's p.Info() logger. master had no
doc comment on handleRecovery, so the v1.0 comment is added. The stop
hook in run() moves to restoreDeviceDNSOnStop; v1.0's router setup and
cleanup hooks next to it are router-only and are left out, since master
removed the router code. fetchGoroutines wraps master's fetchUntilDone,
which takes the metadata function as an argument.
… cleanup, or on a bad config file on mobile

Three paths crashed ctrld, because they used a configuration before
ctrld made sure that it was valid. On mobile, ctrld runs in the process
of the app, so a crash ends the app.

- FirstUpstream and FirstListener return the value of the key with the
  lowest number, or nil when no key is a number. The self-check skips
  the external lookup when there is no first upstream, and fails when
  there is no first listener.
- On mobile, a config.toml or ctrld.toml that ctrld cannot read or
  parse fails the start with UNCLASSIFIED, and the app gets one Exit
  line. Before, Fatal ended the app at each start.
- writeConfigFile replaces the file in one step (temporary file and
  rename, keeping the owner and mode), so an app that ends during a
  write no longer leaves a half-written file behind.
…istener

A listener key that is not a number, or no listener at all, crashed
ctrld. On mobile, ctrld runs in the process of the app, so the crash
ended the app.

- tryUpdateListenerConfig checks the listeners first. With no listener,
  a bad key, or a listener with no settings, the start fails with
  UNCLASSIFIED (exit 59); on mobile the app gets one Exit line.
- validateConfig rejects a listener key that is not a plain number, so
  a reload of such a file is refused and a custom config from the API
  with such a key is disregarded (and its sections are cleared before
  the generated config is built).
- A mobile start with no config file writes the default ctrld.toml into
  HomeDir, not the working directory, and a failed write fails the start
  instead of ending the app.
…eload

serveDNS read p.cfg.Service.InterceptMode with no lock when it started the
local IPv6 listener, and the DNS handler read p.cfg.Service.ClientIDPref
with no lock for each query. A reload writes *p.cfg under p.mu in runWait,
so under -race TestReloadThatDropsARunningListener/control-server-accepted
failed with the write of runWait against the read at dns_proxy.go:315.

serveDNS now reads the listener config and the intercept mode once under
p.mu, before the listener goroutines start, and the handler reads the
client ID preference under p.mu, as it already does for the listener
config.
A fallback-mode Internal Domain whose resolvers cannot be reached (VPN
off or still connecting) ends with the network's own negative answer.
ctrld passed it on with that network's negative-cache lifetime; a home
router forwarding to public DNS gives the root zone's SOA, TTL 3600.
macOS caches the NXDOMAIN for that long, so the name stays broken after
the VPN is back, and the cached lookups never reach ctrld to be fixed.

Cap the SOA TTL and MINIMUM (RFC 2308 section 5) of that negative
answer at 5 seconds. Positive answers, final answers from the
configured resolvers, explicit-only mode and every other name keep
their TTLs.
… without IPv6

On a bridge whose member ports have disable_ipv6=1, ctrld opened an IPv6
mDNS socket on every multicast interface. Writes on the member ports failed
with EACCES, probe() returned the last error even though br-lan sent fine,
and EACCES was not a stop error, so probeLoop retried and warned every
20 to 40 seconds for the life of the process.

- Open IPv6 mDNS sockets only on interfaces with an IPv6 address, as NDP
  discovery already does.
- probe() judges each socket: the round succeeds if any socket sent, and
  sockets with a permanent send error are dropped from later retries.
- EACCES is a permanent send error. probeLoop stops, with one warning,
  only when no socket is left.
… DNS

On Windows in dns intercept mode, a WireGuard full tunnel connected
before ctrld starts blocks DNS to the loopback listener (its kill switch
WFP filters), and the start self-check rolls the install back. ctrld
started first keeps working. Workarounds: start ctrld before connecting,
or use split AllowedIPs so the kill switch stays off.
…tart

At a start in DNS intercept mode ("dns" or "hard") on Windows and macOS,
postRun now resets an adapter's DNS to DHCP only when its static DNS
still names a loopback listener, the setting an earlier interface DNS
mode run or a crash leaves behind. DNS a full-tunnel VPN pushed to its
adapter, or static DNS an administrator set, is left alone. Interface
DNS mode keeps the blanket reset. ctrld restart and the upgrade binary
switch, which clean up DNS in the CLI process after stopping the
service, use the intercept mode the service reports over the control
socket and apply the same rule.
master adaptation: master's SetupTransport takes a context, so the test
calls SetupTransport(context.Background()). The fix applies unchanged:
master's newDOHTransport and newDOH3Transport set the same finalizers,
and its SetupTransport calls the same retireTransports.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants