Skip to content

✨ integrate crosspaste-mouse as a plugin for cross-device mouse sharing#4222

Open
guiyanakuang wants to merge 34 commits into
mainfrom
claude/elastic-hypatia-c325a1
Open

✨ integrate crosspaste-mouse as a plugin for cross-device mouse sharing#4222
guiyanakuang wants to merge 34 commits into
mainfrom
claude/elastic-hypatia-c325a1

Conversation

@guiyanakuang
Copy link
Copy Markdown
Member

Summary

Integrates the crosspaste-mouse Rust daemon as a subprocess plugin of crosspaste-desktop. Users can now share one keyboard/mouse across paired devices and adjust the virtual screen layout by dragging in a new Settings → Mouse panel.

  • IPC layer (Tasks 1–6): spawn crosspaste-mouse plugin, speak its JSON Lines protocol (v2) over stdin/stdout with runInterruptible + NonCancellable for clean shutdown / atomic writes.
  • Lifecycle (Task 7): MouseDaemonManager drives start / UpdateLayout / stop based on mouseEnabled + mouseListenPort config + paired devices (from SyncRuntimeInfo) + the layout store.
  • UI (Tasks 9–12): drag-to-arrange Compose Canvas + enable toggle + permission-warning dialog + i18n (en/zh).
  • Koin wiring (Task 8): DesktopMouseModule registers everything and the manager runs in ioCoroutineDispatcher at app launch (non-headless only).

Trust model: the desktop tells the daemon which peer addresses are valid via already-paired SyncRuntimeInfo rows; no cert fingerprint is sent — crosspaste-desktop's own pairing flow is the trust authority, so the daemon's built-in SkipServerVerification is acceptable here.

The full implementation plan lives in docs/superpowers/plans/2026-04-20-crosspaste-mouse-integration.md.

Known gaps / follow-ups

  • Binary is NOT bundled into the installer. Dev users point at it via CROSSPASTE_MOUSE_BIN env var or -Dcrosspaste.mouse.binary=<path> system property. Production bundling (Conveyor + cargo cross-compile) is a separate plan.
  • Layout changes drop + reconnect QUIC because daemon ticket System Tray Support #9 (UpdateLayout hot-swap) is still pending. Once it lands, MouseDaemonClient already prefers the native update_layout command when advertised via capabilities.
  • Other locales (de/es/fa/fr/ja/ko/pt/zh_hant) fall back to English until translators add the 10 new mouse_settings* keys via i18n_batch_update.sh.

Test plan

  • ./gradlew ktlintCheck clean
  • ./gradlew app:desktopTest --tests "com.crosspaste.mouse.*" --tests "com.crosspaste.ui.mouse.*" — 33/33 passing locally
  • Build the Rust daemon: cd ~/crosspaste-mouse && cargo build --release
  • Launch desktop: CROSSPASTE_MOUSE_BIN=~/crosspaste-mouse/target/release/crosspaste-mouse ./gradlew app:run
  • Pair a second device via the existing pairing flow
  • Open Settings → Mouse, toggle on, grant macOS Accessibility permission when prompted
  • Drag remote device rectangle to the LEFT of local — cursor should cross on left edge
  • Drag remote rectangle to the RIGHT — cursor should cross on right edge
  • Disable → verify daemon stops + state returns to Off

- Raise events SharedFlow buffer to 256 and document DROP_OLDEST policy
  so callers cannot assume every event is delivered.
- close() now interrupts the blocking readLine() via runInterruptible,
  closes stdout, and joins the reader job — so AutoCloseable returns
  only after the reader coroutine is done (PipedInputStream.close()
  alone does not wake a reader parked in read()).
- send() wraps the write+flush in NonCancellable so a mid-send cancel
  cannot leave a half-written JSON line on the daemon's stdin.
- Drop the dead 'if Stopped { }' branch; keep the EOF-only contract as a
  loop comment.
- Fix spawn() KDoc to describe what it actually throws (IOException
  from ProcessBuilder.start, not IllegalStateException).
- Document constructor stream ownership.
Introduce desktopMouseModule() that registers MouseLayoutStore (backed by
DesktopConfigManager via a new typed updateConfig(updater) overload for
Map/complex fields) and MouseDaemonManager (with flows derived from
DesktopAppConfig.mouseEnabled/mouseListenPort).

Launch MouseDaemonManager.run() from CrossPaste startup in non-headless
mode only — the daemon is a GUI-oriented feature and serves no purpose
in headless/server environments, so desktopMouseModule() is also only
added to the non-headless Koin modules list.
Adds en/zh translations for the mouse sharing settings screen and
permission dialog, and replaces hardcoded literals in
MouseSettingsScreen and MousePermissionDialog with GlobalCopywriter
lookups. The nav list entry in AdvancedSettingsContentView already
references mouse_settings / mouse_settings_desc, which are now defined.

Other locales (de/es/fa/fr/ja/ko/pt/zh_hant) still need translations;
i18n_batch_update.sh expects an input file with per-locale values and
was not run here because only en/zh copy is available.
Coroutine cancellation via runInterruptible interrupts readLine(), which
throws InterruptedIOException. Without catching it, the exception leaks
onto the JVM's uncaught-exception queue and breaks adjacent runTest
tests with UncaughtExceptionsBeforeTest — surfaced as a flake when
MouseLayoutStoreTest or ScreenArrangementViewModelTest ran after
MouseDaemonProcessTest in the same JVM fork.
…ns pattern

Drop the superfluous updateConfig(updater) lambda overload on
DesktopConfigManager — it duplicated machinery the scalar path already
has. mouseLayout now flows through the standard scalar copy(key, value)
path: the MouseLayoutStore.Backing adapter JSON-encodes on write and
decodes on read (via Position's existing @serializable), mirroring how
blacklist, sourceExclusions and useNetworkInterfaces are persisted.
upsert/remove previously did snapshot() -> mutate -> set(), allowing
concurrent writers to read the same baseline and overwrite each other
on write (lost-update race). Replace Backing.set() with update((Map) ->
Map) so the read, mutate and write happen atomically inside the backing.

- DesktopAppConfigMouseLayoutBacking wraps the whole update in
  synchronized(updateLock).
- Tests' FakeBacking impls use @synchronized update().
- Add a concurrency regression test: 200 concurrent upserts on distinct
  keys must all survive (would have silently lost most under the old
  design).
…-context deadlock

close() used runBlocking { cancelAndJoin() } to wait for the reader
coroutine before returning. If close() is called from a coroutine
context (e.g. the MouseDaemonManager collector on a constrained
dispatcher, or from inside the reader itself), runBlocking inside a
coroutine can deadlock the caller's thread.

Replace with a non-suspending scope.cancel(). runInterruptible already
translates the cancel into Thread.interrupt() on the reader's own
Dispatchers.IO thread — the reader unwinds asynchronously. We lose
the strict 'no worker thread outlives close()' property, but the
reader has no external side effects beyond _events.tryEmit into a
DROP_OLDEST buffer, so an async teardown is safe.
… out-of-sync risk

Previous design kept a separate _flow MutableStateFlow inside the
backing adapter and updated it after configManager.updateConfig. If
updateConfig failed and internally rolled back _config.value, our
_flow would still advance to the new value — leaving observers in an
inconsistent state relative to the persisted config.

Refactor: make Backing.flow() return Flow (not MutableStateFlow), and
in the desktop adapter derive it directly from configManager.config
via map + distinctUntilChanged. Config is now the single source of
truth — rollbacks in updateConfig propagate to observers automatically.

Test FakeBacking impls declare override return type as Flow<...>
explicitly so the compiler generates the required bridge method at
the JVM level (AbstractMethodError would fire otherwise).
… boundary

The stop+start fallback (used when the daemon does not advertise
update_layout) emits a Stopped between the two commands that is
session replacement, not a real shutdown. Downstream observers
(MouseDaemonManager state machine, UI) must not treat it as the
daemon stopping.

Route all handle.events through run()'s single collector, which:
  - suppresses Stopped while restartInProgress is set,
  - forwards everything else to a private SharedFlow,
  - clears restartInProgress once the new session's Initialized or
    Ready arrives — any Stopped after that is a real shutdown.

Doing the decision at run()'s entry (not with a downstream filter)
removes the race where a lazy filter could be evaluated after the
flag had already been cleared.

Add regression test covering both the suppressed transient and the
subsequent real Stopped.
…g layout

Previously, a corrupt mouseLayout field in appConfig.json would fall
back to emptyMap() with no log output — users would lose their
arrangement with no diagnostic. Add a KotlinLogging warn with the
caught exception and the offending JSON (truncated to 200 chars) so
operators can see what went wrong.

De-duplicated via lastWarnedJson to avoid spam: decode() runs on every
snapshot/update and on every emission of the derived flow, so a
persistently-corrupt value would otherwise log the same error dozens
of times per session.
…updateClientLayout

run() was an ~82 line combine().collect {} lambda mixing three control
paths (disabled teardown, first-enable spawn, layout-change update).
Extract a local Session bundle (client + two Jobs + last inputs) and
three private methods:

  - teardownClient(current): stop + close client, cancel jobs, reset state
  - spawnClient(inputs): create client, wire event forwarding, start
  - updateClientLayout(current, inputs): diff port/peers, call updateLayout

Plus forwardClientEvents() carved out of the inline event-routing
coroutine. run() itself now reads as a four-way state-machine dispatch.

Behavior preserved — all 5 manager tests still pass unchanged.
Method is a test-only injection hook — make it internal so external
consumers of the app module can't reach it while desktopTest (same
module) retains access. The 'For tests' KDoc alone did not enforce
the restriction; the repo has no @VisibleForTesting convention.
….send()

Flag for the next person to read this code: the NonCancellable wrap
was chosen over caller-cancellation semantics to protect the daemon's
JSON-Lines parser from truncated writes. The consequence is that
close() waits on writeLock until any in-flight send finishes. Spell
that out in the KDoc so nobody 'simplifies' it away.
…vider

Adds a synchronous LocalScreensProvider so the canvas can render the
user's monitors even before the mouse daemon has emitted Initialized.
ScreenInfo gains @transient `name` and `wallpaperPath` populated on the
desktop side (NSScreen + wallpaper PNG copy on macOS, AWT fallback
elsewhere) — never crosses the daemon IPC boundary.
…unning

The daemon's plugin mode is supposed to keep stdout pure JSONL, but
log lines (with ANSI escapes) occasionally leak there. Treat any line
not starting with '{' as log spillover — log it on our side, but don't
turn it into IpcEvent.Error and surface a red banner in the UI.

Also map IpcEvent.Initialized / Ready to MouseState.Running so a
freshly-started daemon doesn't sit in Starting forever and any
prior transient warning gets cleared.
Replaces the in-app navigation entry with a standalone Compose Window
launched from Extensions. The window is resizable and has a 600x500 px
minimum size enforced on the underlying AWT window so it can grow to
fit denser layouts without ever shrinking past usable. Visibility is
driven by a new mouseSettingsWindowVisible state on
DesktopAppWindowManager.

Removes the now-orphaned commonMain MouseSettings route, the duplicate
entry in AdvancedSettings, and the SettingsGraph composable
registration so navigation no longer reaches the in-app screen.
MouseSettingsScreen now fills the whole window so the canvas grows
with it (both width and height). The canvas itself adds Modifier
.clipToBounds() so screen rectangles, wallpapers, and labels at
extreme pan/zoom can no longer bleed onto neighbouring UI.

Pan and zoom now respect a world-bounding-box-aware clamp:

- Pan keeps at least 80 px of bounds visible against every viewport
  edge so the layout can never drift entirely off-canvas.
- Zoom-out floor is fixed at 50 dp per reference 1080p screen via
  LocalDensity, independent of window size — maximising the window
  no longer pushes the floor up to fit-the-bounds.
- Zoom-in ceiling stays at 1:1.

Viewport dimensions now come from BoxWithConstraints.constraints
(px) instead of maxWidth/maxHeight (dp). The dp-as-px mismatch had
been silently halving the right/bottom pan reach on HiDPI screens.

Screen labels rendered with TextMeasurer: font size scales with the
rect's smaller side, text is wrapped/ellipsised to fit inside the
rect, and a translucent rounded backdrop keeps it legible over any
wallpaper.
…erage

Renames the entry label in en/zh from "Mouse" / "鼠标共享" to
"Keyboard & Mouse" / "键鼠共享" — the feature shares keyboard and
mouse, and the surrounding strings already reflected that.

Adds the full mouse_settings key set (10 entries: title, desc,
canvas help, 5 state strings, switch label, dialog OK) for de, es,
fa, fr, ja, ko, pt, and zh_hant; previously they fell back to en.
@guiyanakuang guiyanakuang force-pushed the claude/elastic-hypatia-c325a1 branch from 4f257b9 to 5e5fe2c Compare April 26, 2026 05:33
The DAO previously used a Channel<String>(UNLIMITED) to notify
getAllSyncRuntimeInfosFlow subscribers of writes. Channel.receiveAsFlow
has fan-out semantics — each emission is consumed by exactly one
collector — so adding a second subscriber caused them to race for each
event. About half the time the wrong collector won, leaving the
SyncManager-driven UI/sync state stale (e.g. removed devices not
disappearing from the list).

Switch to a MutableSharedFlow<Unit> "table changed" signal: every
subscriber sees every change. Subscribers re-read the full table on
each tick via onSubscription { emit(Unit) }.map { getAllSyncRuntimeInfos() },
which also gives them an initial snapshot without a separate code path.

Add a regression test that pins the multi-subscriber behavior.
MouseDaemonManager was reading the paired-device list directly from
SyncRuntimeInfoDao.getAllSyncRuntimeInfosFlow(). That works in
isolation, but it bypasses the canonical multi-cast view exposed by
SyncManager.realTimeSyncRuntimeInfos and conceptually mouse cares about
"business peers managed by SyncManager", not raw table rows.

Replace the SyncRuntimeInfoDao constructor dependency with a
Flow<List<SyncRuntimeInfo>> and wire it to
SyncManager.realTimeSyncRuntimeInfos in DesktopMouseModule. Tests pass
the same flow shape, so the fake-DAO mock is no longer needed.
The previous default candidate `$HOME/crosspaste-mouse/target/release/crosspaste-mouse`
was a developer-machine convention that breaks on Windows (no `.exe` suffix)
and has no defined production location.

- Read the dev binary path from a new `mouseBinaryPath` key in
  `development.properties` (added to the tracked template).
- In production, look at `pasteAppExePath / crosspaste-mouse[.exe]` —
  add `MouseDaemonBinary.binaryName(platform)` for the `.exe` suffix.
- Move env-aware candidate assembly into the caller (`DesktopMouseModule`)
  so `MouseDaemonBinary` stays a generic resolver.
- Make the "binary not found" error environment-aware: dev points at
  `development.properties`, prod surfaces the attempted bundled path
  and tells the user to reinstall.
- Keep `-Dcrosspaste.mouse.binary` and `$CROSSPASTE_MOUSE_BIN` as
  per-launch / per-shell escape hatches.

Closes #4303
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant