Compare commits

..
Author SHA1 Message Date
The412Banner cf6cf168b0 feat: x86-64 client follows DroidDeck-Components' fex.json; fuller x86-64 session logs (#1053)
FEX builds can reach devices without an app release: while the x86-64
client is selected the app reads DroidDeck-Components' fex.json (directly,
on its own thread, not in Offline Mode) and a session passes that build to
droiddeck-steam-x64, which installs it only once its sha256 matches and its
programs load, keeping the replaced files aside until the installed copy
loads too; it keeps a working installed build otherwise and falls back to
the build it pins. FEX is built and published by Components' build-fex.yml
on its single fex release; this repository's build-fex.yml only checks the
build. x86-64 session logs name the FEX build and its source, the rootfs and
the client's builds, report the rootfs download in tenths, and say in one
line when the web helper's GPU process keeps exiting, with x86 glxinfo and
vulkaninfo through FEX. Settings shows the FEX build installed.
2026-10-18 03:40:17 -04:00
The412Banner 7d21ba2233 fix: x86-64 Steam client fails without the KDE desktop (FEX r5 bundles libxxhash) (#1050)
FEX r4 links libxxhash, which the Linux runtime does not ship; devices
without the KDE desktop package (where the library came from) exited 127
at the x86-64 client's first FEX call. FEX r5 bundles it beside libFEXCore
with RUNPATH $ORIGIN/../lib, and the FEX build now fails when a program in
the package needs a library that is neither bundled nor listed in
tools/fex/runtime-libs.txt. droiddeck-steam-x64 moves to fex-2610-r5
(DroidDeck-Components #5). Progress log: the org review of #152 and this fix.
2026-10-18 02:00:29 -04:00
The412Banner ae94d9db54 feat: optional x86-64 Steam client (Valve's Linux client under FEX) beside the arm64 one (#152)
An opt-in second Steam client (Steam settings, Steam client: x86-64): Valve's
x86-64 Linux client, unchanged, runs under FEX in its own home with its own
game library and the same sign-in. Linux builds run directly, so titles such
as Team Fortress 2 and Half-Life use their Linux versions; Windows games use
DroidDeck-Proton as on arm64. The arm64 client stays the default.

- droiddeck-steam-x64: FEX and the x86-64 rootfs from DroidDeck-Components
  (sha256-pinned), client install and sync from Valve's CDN, sign-in shared
  with the arm64 client, its web helper on the GPU (ANGLE GL over FEX's GL
  thunk, zink, Turnip), Proton wrappers for the arm64 client's tools,
  DroidDeck direct for Linux builds, SD-card library (x86-64/ beside the
  arm64 one), repair.
- FEX 2610 built by build-fex.yml with thunks and three patches: an openat2
  fallback without openat2 (proot), newer-clang thunkgen, MAP_32BIT
  exhaustion returning ENOMEM. libblsession resolves RESOLVE_IN_ROOT paths
  inside the root.
- Session: x86-64 gating, shortcuts and compatibility tools per client,
  logs from the client that ran.
- App: client setting with the FEX version, an x86-64 chip on the Games
  header, the active client's library, art, storage, downloads, repair,
  Offline Mode, Decky, desktop mode, optional VAC diagnostics; per-client
  state for art, the content log, repair and leftovers.
- Strings in every locale.
- Review fixes: the library cache is saved only by the newest scan; an
  abandoned session's Steam logs come from the client it recorded; distinct
  VAC console names; FEXInterpreter in CPU pinning; clearer VAC hint in
  pl, ru, uk, cs and bg.
2026-10-17 23:57:52 -04:00
MaxsTechReview 45513c90d6 fix: rename the VR recovery task affinity in variant builds (#1038) 2026-10-17 17:24:31 -04:00
MaxsTechReview f3f3ba1a8a fix: rename the VR recovery task affinity in variant builds
The PUBG, AnTuTu and Ludashi signing jobs stopped on
com.droiddeck.launcher.vr_recovery, the task affinity of the VR
recovery activity added with the VITURE XR controls, because the
package rename did not cover it.
2026-10-17 17:14:06 -04:00
MaxsTechReview 97f976ef6c feat: Epic Games Launcher for Linux (#928)
* feat: Epic Games Launcher for Linux

Install Epic's official Linux launcher (EGLv2) from Epic's CDN with Valve's
Steam Runtime 4 and run it under FEX, signed in with the linked Epic account.
Epic games launch through a hidden launcher instance so they get the
launcher's own arguments, with the exchange-code launch as the fallback, and
DroidDeck installs show up in the launcher as installed.

In the launcher, touch scrolls like a phone app and the gamepad drives the UI:
D-pad and left stick move the selection, A opens, B goes back, the right stick
and triggers scroll proportionally, X opens the options menu, Y the keyboard
and LB/RB switch tabs.

* feat: Epic Games Launcher for Linux

Install Epic's official Linux launcher (EGLv2) from Epic's CDN with Valve's
Steam Runtime 4 and run it under FEX, signed in with the linked Epic account.
Epic games launch through a hidden launcher instance so they get the
launcher's own arguments, with the exchange-code launch as the fallback, and
DroidDeck installs show up in the launcher as installed.

In the launcher, touch scrolls like a phone app and the gamepad drives the UI:
D-pad and left stick move the selection, A opens, B goes back, the right stick
and triggers scroll proportionally, X opens the options menu, Y the keyboard
and LB/RB switch tabs.

* Route Epic games through Steam, import launcher installs, offer the second library

- Every Epic launch goes Steam -> Epic Games Launcher -> game; Play inside the open launcher hands the game to Steam
- Games installed from inside the launcher get a Games-tab entry and launch through Steam; the launcher installs to the last Stores target
- Stores install picker offers the Steam second library, with side-by-side choices
- uruntime xattr-only failures no longer fail the launcher unpack (app-extract)

* Hand launcher Play and Steam links to Steam, wait for FEX on first run

Play on a game the launcher just installed is matched to its install
record and handed to Steam. Steam games listed in the launcher open
steam://rungameid links, which now start the game in a Steam session.
Opening the launcher before Steam has downloaded FEX asks Steam for it
and starts Steam instead of failing with no-fex.

* Address review: FEX check, play-in-Steam action, touch route thread, scroll scale

fexReady needs a FEX binary in Steam's FEX tool, not just its folder.
The play-in-Steam action stays until the link is handled, so Run Steam
at startup can't start a session first. The pad touch route reads the
mouse mode without side effects off the main thread and posts the pad
update there. Vertical scroll uses the output height. A status-2 unpack
counts only when AppRun is there. The xdg-open command is tested
end to end for Steam links.

* Open the launcher through Steam, recover dropped shortcuts, sign out of both

The Epic Games Launcher now runs as its own Steam shortcut, so opening
it starts Steam, which opens the launcher, and Play inside it goes to
the running Steam.

Steam dropped a running shortcut from its library when the game exited
after a sleep/wake replay or a server re-logon, and launching it then
waited forever at ShowInterstitials. login1 only replays sleep/wake
when Steam is asleep. The session restarts Steam when a requested
shortcut is missing, and when a game exits with its shortcut gone and
nothing else runs.

Resuming after a long pause no longer reports failure while traced
processes are still waking, so "Steam is paused" goes away on the first
tap.

Signing out of the launcher signs DroidDeck's Epic store out, and
signing out in DroidDeck restarts the launcher signed out and clears
the Epic web cookies.

* Address review: one session-switch poll, Epic setup can't block a session

A new switch confirmation replaces the pending poll instead of adding a
second one that could start the session again. Epic library sync and
install-folder setup at session start are each guarded, so a bad
preference no longer stops the guest from launching.

* Uninstall the Epic Games Launcher from the Games tab

An Uninstall button next to Launch asks first, closes a running
session, and removes the launcher and its runtime. The launcher's
sign-in and install records stay, so its games keep their Games-tab
entries and a reinstall comes back signed in. Its Steam shortcut goes
away on the next Steam start.

* Address review: uninstall only after the session has ended

The launcher's files are removed only once the session is not running
and its phase has ended; a shutdown still in progress after the wait
reports failure instead. The confirmation reads the session state from
the front-end state.

* Find FEX that Steam installed on the second library

Steam's tools on a second library live in DroidDeck's private copy,
which the session binds over the library. The launcher's FEX check
only looked on the shared storage, so with FEX there it always asked
Steam for FEX and only Steam opened. It now checks the private copy
too.

* Start games without Steam, per-game Settings menu with Wine sync

"Start without Steam" now really leaves the Steam client out: the game
runs in its own Proton/FEX through droiddeck-proton-run in a run
session, and the Epic Games Launcher opens on its own with its games
playing in it. Run sessions for these games are not pinned to the big
cores and get DirectAudio and the added-games list like Steam.

The Games tab card is now "Settings", titled "<game> Settings": Start
without Steam, FEX preset, Wine sync (ntsync, fsync, esync, wineserver
or the app default) and Default, which puts everything changed for the
game back to the app's settings. The app-wide Wine sync choice is shown
in the Steam settings Session tab.

* Address review: uninstall guard, routed games, shortcut matching

Opening or reinstalling the Epic Games Launcher waits while it is being
uninstalled. Added games that are Steam installs keep their Start
without Steam setting under their Steam app id and start as that Steam
game. Live shortcut lookups wait for Steam's library to load and match
a same-name shortcut by its executable, so no duplicate is added. The
session treats the open launcher as itself even when it could not tell
its Steam app id.

* Address review: drop a queued launcher switch when it is uninstalled

A confirmed switch to the Epic Games Launcher that is still waiting for
the old session to end is cancelled when the launcher is uninstalled,
and never starts while the uninstall runs. The launcher id test sets
its environment for its own runs only.
2026-10-17 14:40:39 -04:00
+13 83eda433c8 Add VITURE XR controls, head tracking, and SteamVR (#732)
* Add VITURE glasses support: hardware control, 3D, head tracking

Glasses get their own controller over the MCU (permission, 120 Hz default, brightness, volume, sunblock, 2D/3D) and the game moves onto them when plugged in. The IMU drives Cardboard head tracking and the SteamVR driver. 3D falls back to one eye's view on phones that can't send the 3840x1080 signal. Options open from a glasses button in game and on the Steam page.

* Drop the glasses settings like Steam's: Play-style button, Settings lip, tabs

* Hang the glasses Settings lip over the top of the button

* Space glasses MCU commands so the IMU rate sticks, fit fixed resolutions to the glasses

* Keep Steam alive with SteamVR registered, run SteamVR at the glasses' rate, keep the glasses lip on top

* Fit VR games on the glasses: full view, seated origin at the head, pad hands reach with the d-pad

* Let the pad hands reach a metre out so buttons with deep colliders press

* Show every VR frame in order at 120 fps: wait for SteamVR's GPU work, present only finished copies, skip the grace on idle shared counters

* Glasses: SteamVR starts and stops with them, closer view with a field of view setting, Recenter on the pad

* Glasses: SteamVR field of view defaults to 90°

* VR pad hands: sticks move and aim the hands, left stick click switches the sticks to the game, d-pad pulls a held hand

* Glasses: Play SteamVR beside the glasses button instead of starting SteamVR with them, pad hands drawn as Index controllers and off while other VR controllers are connected, glasses sheet tabs sized like Steam's

* Track glasses mode, refresh-aware glasses window, align Play SteamVR with Play Steam

* Fix 3D on the glasses: full-res SteamVR presenter, revert VR 3D at session end, coalesce vsync ticks, no session restart after end

* Address review: glasses output after installs and session end, VR revert retry, display fallback, presenter swapchain retry, agent hex checks, drawer bumpers, glasses translations, docs

* Harden VITURE stereo negotiation and output refresh tracking

* Localize stereo recovery feedback and record Astra reconnect tests

* Improve test failure diagnostics and refresh Astra firmware findings

* Handle VITURE docks separately and reject truncated USB frames

* Glasses: remember a 3D fallback per model, docks as display only, VR timing fixes

- A 3D attempt that ends at the 640x480 fallback is remembered for that glasses
  model, so Play SteamVR no longer blacks out the glasses after every restart;
  Retry clears it and "3D for VR games" stays reachable
- A VITURE Mobile Dock on its own no longer counts as glasses; through a dock
  Play SteamVR is hidden and the sheet shows the output and why there is no control
- HDR is probed on the glasses when the session starts there
- Glasses frames with a mismatched checksum are marked in the status frame log
- SteamVR layer and driver vblank counters stay monotonic across refresh changes;
  one-eye render scale applies from activation and on mono/3D switches
- Compositor binds announce the refresh clients were last told; the presenter
  retries a failed first swapchain
- Dead strings and helpers removed; docs updated

* Glasses: drop USB frames with a bad checksum

* Glasses: count frames dropped for a bad checksum

* Feat/viture xr (#907)

* Add RayNeo Air and GT support to the XR glasses pipeline

* Add RayNeo Air and GT support to the XR glasses pipeline

* Checkpoint glasses tracking, calibration, and SteamVR launch integration

* Checkpoint glasses tracking, calibration, and SteamVR launch integration

* Add XREAL 1S support and correct GT and Pro 2 tracking

* Add XREAL 1S support and correct GT and Pro 2 tracking

* Fix mirrored glasses tracking and recover SteamVR after device loss

* Fix mirrored glasses tracking and recover SteamVR after device loss

* Document confirmed XREAL 1S brightness control

* Document confirmed XREAL 1S brightness control

* Fix SteamVR glasses defaults and rendering controls

* Fix SteamVR glasses defaults and rendering controls

* chore: keep only the 3 newest session log folders (#852)

* chore: add agent comment rules (#748)

Co-authored-by: Kurt Himebauch <[email protected]>

* Fix SteamVR polling, VITURE checksums, and fence tracking

Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter.

Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage.

* Fix SteamVR polling, VITURE checksums, and fence tracking

Keep manual SteamVR launches polled, discard corrupt VITURE packets, and retire Vulkan fences out of order without blocking submissions at a fixed capacity. Preserve the contiguous completion watermark used by the presenter.

Fix optional Windows launcher configuration and audited helper staging. Add Android, Windows, and native tracker regression coverage.

* fix: wireless ADB page names the child-process switch the way Android shows it (#848)

* l10n: refresh stale and untranslated strings in es, fr, ko, ru (#850)

* l10n: refresh stale and untranslated strings in es, fr, ko, ru

- mode_added_art_hint: now mentions Steam then SteamGridDB (all four locales).
- wincomp_needs_installer / wincomp_installers_note: es and ko still described
  the old "coming soon" behaviour; align with the current English text.
- summary_remove_game: was still English in all four locales.

Machine-assisted translation; native-speaker review is welcome.

* l10n(es): say registro, not grabación, for the missing installer record

---------

Co-authored-by: ovurrsl <[email protected]>

* fix: a failed cloud save recovery no longer ends the app (#846)

The cloud-recovery thread started at every app start had nothing to
catch an error, so one went to CrashHandler, which halts the process
with status 2. In unit tests the thread can outlive the test that
started it, fail on the torn-down preferences, and take the whole test
run down. The error is now logged and the thread ends.

Co-authored-by: Kirk <[email protected]>

* fix: an interrupted Linux runtime download resumes instead of starting again (#847)

* fix: an interrupted Linux runtime download resumes instead of starting again

installOnce() asked the downloader to resume but always deleted the archive in its finally block, so the next attempt began at zero. The archive is now named after the release version and checksum, a failed download keeps what it fetched, and archives of other releases (and the old fixed name) are removed. A checksum or extraction failure still deletes the file.

* test: Downloader resumes a cut-off download from the bytes it kept

Starts a socket server that promises the whole file, hangs up part way and then answers a Range request, and checks that the second downloadFile call resumes with the right offset and ends with identical bytes.

* fix: a resumed download discards a 206 that does not start where the kept bytes end

Downloader appended every 206 response to the partial file without
looking at Content-Range. A server or proxy that answers a Range request
with a different offset would leave a corrupt file. Check that the range
starts at the kept length; otherwise delete the partial file and start
again without a Range header.

Adds a test with a server that answers the Range request from offset 0.

---------

Co-authored-by: ovurrsl <[email protected]>

* l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings (#849)

* l10n: German, French, Spanish, Italian, Portuguese and Russian for the zRAM, reset-settings and store free-space strings

Adds the strings values-de, values-it and values-pt were missing (14 each) and the six that values-fr, values-es and values-ru were missing. Machine-assisted drafts that follow each file's existing register and terms; native review is welcome.

* l10n: correct missing active swap translations

---------

Co-authored-by: ovurrsl <[email protected]>
Co-authored-by: xXJsonDeruloXx <[email protected]>

* fix: droiddeckctl reaches variant-package builds (recover #842) (#854)

fix: droiddeckctl reaches builds shipped under a variant package

The provider authority and the start Activity's component were fixed to
com.droiddeck.launcher, so droiddeckctl could not talk to the antutu,
ludashi or pubg builds from tools/release/variants.txt. DROIDDECK_PACKAGE
now picks the package; the Activity's class name stays the same in every
variant.

Co-authored-by: angelrenefm <[email protected]>

* feat: droiddeckctl reports on-screen pacing (recover #843) (#855)

feat: droiddeckctl pacing reports the frames that reached the screen

Reads SurfaceFlinger's timestats for the game's layer (or the session's
SurfaceView) over a capture: frames on screen, fps on screen, the
present-to-present histogram and how many intervals were off the usual
cadence. Records split by a refresh-rate change are summed. A game's own fps
counter counts frames handed over, so it can't show dropped or doubled frames.

Co-authored-by: angelrenefm <[email protected]>

* fix: Steam can restart more than twice in a session (#754)

* fix: Steam can restart more than twice in a session

Every exit 42 counted toward the cap that stops an update which will
not install, and the count was never cleared, so the third restart of
a session ended it however far apart they were. A client that ran for
a minute before asking is not that loop: the count starts again.

* fix: measure the client's run time as it exits

The time spent collecting logs and checking FEX after the client exits
counted toward its run, so a slow cleanup could make a quick exit 42
look like a long one and clear the count.

---------

Co-authored-by: Kirk <[email protected]>
Co-authored-by: Kurt Himebauch <[email protected]>

* fix: the front end keeps its page when the Fold is folded or unfolded (#832)

* fix: the front end keeps its page when the Fold is folded or unfolded

MainActivity, FileManagerActivity, FilePickerActivity and StoreLoginActivity did not declare the screen-size configuration changes, so folding or unfolding a foldable recreated them and dropped their in-memory state. They now handle orientation, screenSize, screenLayout, smallestScreenSize and density themselves, like SessionActivity already does. uiMode is left out on purpose.

* fix: store login keeps rebuilding its toolbar when the display density changes

StoreLoginActivity builds its toolbar in code with dp-to-pixel values taken at creation time and has no onConfigurationChanged. It no longer declares density in configChanges, so a density change recreates it as before this PR. Size and layout changes are still handled in place.

---------

Co-authored-by: ovurrsl <[email protected]>

* Release pointer buttons without replaying stale motion (#803)

* Fix cursor warping on native touch down

* Preserve input cleanup without replaying stale pointer motion

---------

Co-authored-by: siyuan.qsy <[email protected]>

* l10n: translate the strings added since #284 into zh-CN, zh-HK, zh-TW and ja (#539)

* l10n: translate the 184 strings added since #284 (zh-CN/HK/TW, ja)

The stores, cloud saves, Windows components, Steam touch, desktop
removal and background-downloads strings landed after #284's
translation snapshot, so every one of them fell back to English.
Translate all 184 into Simplified Chinese, Traditional Chinese
(Taiwan and Hong Kong) and Japanese, keeping each locale file's own
vocabulary (工作階段/會話, 解除安裝/卸載, 元件/組件, 設定).

Also fix four Hong Kong strings that kept the simplified 着色器.

* l10n(zh): fix stale and ambiguous strings found in review

Severe: summary_remove_game was untranslated English in all three
files; mode_added_art_hint, wincomp_installers_note and
wincomp_needs_installer still described the pre-#480/#492 English.

Also: distinguish the Setup wizard from Android system settings in
zh-HK/zh-TW (初始設定 / 系統設定 / Android 設定), unify library as
收藏庫, component as 組件 in zh-HK, session as 會話 in zh-HK, account
as 帳戶 in zh-HK, Trending as 趋势/趨勢, Forget as 清除, reword the
re-applied-at-launch, action-failed and already-in-Games strings, and
play as 執行/開啟遊戲 in zh-TW/zh-HK.

* l10n(ja): fix stale and ambiguous strings found in review

---------

Co-authored-by: Dantalian <[email protected]>
Co-authored-by: xXJsonDeruloXx <[email protected]>

* Rebuild and verify Windows VR runtime components

* Rebuild and verify Windows VR runtime components

* Add top, bottom, and centered picture positions for Steam sessions (#871)

* fix: dependency scan ignores folders above the game folder (#859)

* DroidDeck-Proton: get releases from the Droid-Deck org repo

DroidDeck-Proton is now built and released on Droid-Deck/DroidDeck-Proton
(11-9.1 is Latest there), so the Components page, the update check and the
guest installer read releases from the org repo instead of The412Banner's.

* feat: low-latency controller and touch toggles in the session drawer (#808)

* fix: SteamGridDB lookups no longer stick as "no match" after a failure; add a search to the art chooser (#876)

* feat: Steam home wall fills from the signed-in library instead of store picks (#878)

* test: cover GPU family and support classification (#880)

* Fix local builds on Linux failing after the first Comet build (#866)

* feat: sort the store library and installed tabs, and switch between card and list views (#798)

* feat: three-finger tap toggles Steam's on-screen keyboard in game mode (#782)

* fix: keep Quest-patched Turnip as the auto default (#853)

* Fix deploy_local.sh and droiddeckctl not finding a phone connected over wireless debugging (#864)

* fix: keep proot sessions alive when the kernel loses fork events (#457)

* fix(quest): correct desktop and Steam display corruption across sessions (#733)

fix(quest): stabilize desktop and Steam layer presentation

Keep Quest SDR scenes and the black backdrop off the Android Vulkan swapchain regardless of session order or HDR preferences. Only report the first frame after client content is submitted.

Add backdrop lifecycle and first-frame regression coverage. Record the user's successful Quest 3 Test 2 result.

Co-authored-by: xXJsonDeruloXx <[email protected]>

* fix: keep SteamVR syscall hooks and report Steam crashes (#905)

* fix: preserve SteamVR syscall hooks

* fix: report Steam crashes through gamescope

* fix: honor glasses pacing and resolution in SteamVR launches

---------

Co-authored-by: connor g <[email protected]>
Co-authored-by: Kurt Himebauch <[email protected]>
Co-authored-by: Yassine Gherbi <[email protected]>
Co-authored-by: Resul Övür <[email protected]>
Co-authored-by: ovurrsl <[email protected]>
Co-authored-by: Kirk <[email protected]>
Co-authored-by: Kirk <[email protected]>
Co-authored-by: xXJsonDeruloXx <[email protected]>
Co-authored-by: angelrenefm <[email protected]>
Co-authored-by: asdf17128 <[email protected]>
Co-authored-by: siyuan.qsy <[email protected]>
Co-authored-by: Dantalian <[email protected]>
Co-authored-by: Dantalian <[email protected]>
Co-authored-by: The412Banner <[email protected]>
Co-authored-by: The412Banner <[email protected]>
Co-authored-by: FX1922 <[email protected]>
Co-authored-by: Muaz Kadan <[email protected]>
Co-authored-by: kingokksa <[email protected]>
Co-authored-by: Jamie Schenker <[email protected]>
Co-authored-by: Jagger15 <[email protected]>

* Glasses: keep a pending 2D restore and the SteamVR recovery offer, show Play SteamVR only with a picture

* Translate the glasses calibration, XREAL and SteamVR recovery strings

* SteamVR recovery: give a finished teardown time to collect logs before restarting

* Credit the projects the XR glasses support builds on

* Joy-Cons: SteamVR hands from their gyro while SteamVR runs, one combined pad otherwise

* Headsets: settings badge on Play SteamVR instead of a top-right button, both shown only once connected

* Hand tracking: camera-tracked Joy-Cons and bare hands in SteamVR, drawn as white gloves

- Joy-Con movement from the accelerometer (report reordering, per-Joy-Con calibration, zero-velocity updates)
- MediaPipe pose and hand landmarks from the device camera, placement check, per-hand Kalman filter with the Joy-Con
- No controller connected: bare camera hands with Quest-style gestures, 1€ filtered, rendered as a white glove model built from a distance field
- SteamVR safe mode cleared before each start so one crash no longer blocks the driver

---------

Co-authored-by: MaxsTechReview <[email protected]>
Co-authored-by: informalTechCode <[email protected]>
Co-authored-by: connor g <[email protected]>
Co-authored-by: Kurt Himebauch <[email protected]>
Co-authored-by: Yassine Gherbi <[email protected]>
Co-authored-by: Resul Övür <[email protected]>
Co-authored-by: ovurrsl <[email protected]>
Co-authored-by: Kirk <[email protected]>
Co-authored-by: Kirk <[email protected]>
Co-authored-by: xXJsonDeruloXx <[email protected]>
Co-authored-by: angelrenefm <[email protected]>
Co-authored-by: asdf17128 <[email protected]>
Co-authored-by: siyuan.qsy <[email protected]>
Co-authored-by: Dantalian <[email protected]>
Co-authored-by: Dantalian <[email protected]>
Co-authored-by: The412Banner <[email protected]>
Co-authored-by: The412Banner <[email protected]>
Co-authored-by: FX1922 <[email protected]>
Co-authored-by: Muaz Kadan <[email protected]>
Co-authored-by: kingokksa <[email protected]>
Co-authored-by: Jamie Schenker <[email protected]>
Co-authored-by: Jagger15 <[email protected]>
2026-10-17 13:17:05 -04:00
kingokksa e95a4226eb l10n: translate the missing strings in every locale, and complete es-419 (#1013)
* l10n: translate the missing strings into zh-CN, zh-HK and zh-TW

* l10n: translate the missing strings into ja and ko

* l10n: translate the missing strings into it, pt, pt-PT, ru and tr

* l10n: translate the missing strings into nl, pl, sv, da, nb and fi

* l10n: translate the missing strings into ar, bg, cs and el

* l10n: translate the missing strings into hi, hu, in, ms and ro

* l10n: translate the missing strings into th, uk and vi

* l10n: complete the es-419 translation

* l10n: translate the missing strings into de, es and fr
2026-10-17 11:44:59 -04:00
FX1922 26ca374c47 feat: Adjust a game’s logo position and size in the artwork editors (#1004)
* Add an Adjust logo screen to move and resize a game's logo in Steam

* Report a failed write of the local logo position instead of throwing

* Record a Steam-side logo position change only once it has been taken over here
2026-10-17 10:02:22 -04:00
FX1922 64d78aa812 Feat/store game artwork (#976) 2026-10-17 01:19:02 -04:00
Resul Övür 522e755b37 feat: the session report lists the panel refresh modes, thermal state and hint-session support (#990) 2026-10-17 02:31:17 +00:00
372 changed files with 50317 additions and 1195 deletions

No files matched your search

+7
View File
@@ -2,3 +2,10 @@ tools/linuxfs/** text=auto eol=lf
tools/mangoapp/** text=auto eol=lf
tools/comet/LICENSE whitespace=-blank-at-eof
tools/comet/droiddeck.patch whitespace=-blank-at-eol,-blank-at-eof
# These helpers are packaged verbatim and executed inside the Linux runtime.
tools/linuxfs/overlay/** text eol=lf
tools/linuxfs/desktop/** text eol=lf
tools/mangoapp/mangoapp text eol=lf
app/src/main/assets/desktop-removal/*.json text eol=lf
app/src/main/assets/desktop-removal/*.txt text eol=lf
+49
View File
@@ -0,0 +1,49 @@
name: Build FEX (x86-64 Steam client)
# Checks tools/fex: the same build DroidDeck-Components' build-fex.yml runs from this repository to publish FEX
# (its single `fex` release and fex.json, which the app follows). Nothing is published from here.
on:
push:
paths:
- 'tools/fex/**'
- '.github/workflows/build-fex.yml'
workflow_dispatch:
inputs:
thunks:
description: 'Build the host thunks (Vulkan/GL/X11/... to the runtime arm64 libraries)'
required: false
default: 'false'
cxxflags:
description: 'Extra CXXFLAGS for a diagnostic build (e.g. -DDEBUG_STRACE)'
required: false
default: ''
permissions:
contents: read
jobs:
build:
runs-on: ubuntu-24.04-arm
timeout-minutes: 120
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Build in Arch Linux ARM
# Inputs reach the shell as environment variables, never pasted into the script.
env:
FEX_EXTRA_CXXFLAGS: ${{ inputs.cxxflags }}
FEX_THUNKS: ${{ inputs.thunks == 'true' && '1' || '0' }}
run: |
set -euo pipefail
docker run --rm -e FEX_EXTRA_CXXFLAGS="$FEX_EXTRA_CXXFLAGS" -e FEX_THUNKS="$FEX_THUNKS" -v "$PWD:/work" -w /work menci/archlinuxarm:base-devel bash tools/fex/build-in-arch.sh
sudo chown -R "$(id -u):$(id -g)" .
- uses: actions/upload-artifact@v4
with:
name: fex
path: |
fex.tzst
fex.tzst.sha256
fex-needed.txt
fex-files.txt
+65 -17
View File
@@ -26,7 +26,7 @@ concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
PRELOAD_CACHE_VERSION: v3
PRELOAD_CACHE_VERSION: v6
jobs:
build:
@@ -313,9 +313,15 @@ jobs:
set -euo pipefail
docker run --rm -v "$PWD:/src" -w /src public.ecr.aws/docker/library/debian:bullseye bash -c '
set -euo pipefail
printf "deb http://archive.debian.org/debian bullseye main\ndeb http://archive.debian.org/debian-security bullseye-security main\n" > /etc/apt/sources.list
apt-get -o Acquire::Check-Valid-Until=false update -qq
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends gcc g++ gcc-multilib g++-multilib binutils >/dev/null
install_compilers() {
apt-get -o Acquire::Check-Valid-Until=false update -qq &&
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends gcc g++ gcc-multilib g++-multilib binutils >/dev/null
}
printf "deb http://deb.debian.org/debian bullseye main\ndeb http://security.debian.org/debian-security bullseye-security main\n" > /etc/apt/sources.list
if ! install_compilers; then
printf "deb http://archive.debian.org/debian bullseye main\ndeb http://archive.debian.org/debian-security bullseye-security main\n" > /etc/apt/sources.list
install_compilers
fi
tools/linuxfs/build-x86-preloads.sh app/src/main/assets/linuxfs
'
sudo chown -R "$(id -u):$(id -g)" app/src/main/assets/linuxfs
@@ -506,7 +512,24 @@ jobs:
run: python3 -m unittest discover -s tools/tests -p 'test_*.py'
- name: Unit tests
run: ./gradlew testReleaseUnitTest --console=plain -PndkVersion="$NDK_VERSION"
id: unit-tests
run: |
set -o pipefail
mkdir -p app/build
./gradlew testReleaseUnitTest --console=plain --info --stacktrace -Drobolectric.logging=stderr -PndkVersion="$NDK_VERSION" 2>&1 | tee app/build/unit-tests.log
- name: Upload unit test diagnostics
id: unit-test-diagnostics
if: always() && steps.unit-tests.outcome != 'skipped'
uses: actions/upload-artifact@v4
with:
name: unit-test-results
path: |
app/build/unit-tests.log
app/build/test-results/testReleaseUnitTest/
app/build/reports/tests/testReleaseUnitTest/
if-no-files-found: ignore
retention-days: 14
- name: Sign with the testkey (v1 + v2 + v3)
run: |
@@ -536,6 +559,15 @@ jobs:
grep -qi "CN=Android, OU=Android, O=Android" /tmp/sig.txt \
|| { echo "::error::apk is not signed with the AOSP testkey"; exit 1; }
- uses: actions/upload-artifact@v4
id: apk-upload
with:
name: droiddeck-apk
path: app/build/outputs/apk/release/*.apk
if-no-files-found: error
retention-days: 30
compression-level: 0
- name: Report on the pull request
if: always() && github.event_name == 'pull_request'
continue-on-error: true
@@ -544,6 +576,9 @@ jobs:
PR: ${{ github.event.pull_request.number }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
BUILD_OUTCOME: ${{ steps.build.outcome }}
UNIT_TEST_OUTCOME: ${{ steps.unit-tests.outcome }}
TEST_DIAGNOSTICS_URL: ${{ steps.unit-test-diagnostics.outputs.artifact-url }}
APK_OUTCOME: ${{ steps.apk-upload.outcome }}
SAME_REPO: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
run: |
set -uo pipefail
@@ -554,8 +589,8 @@ jobs:
echo
[ -f /tmp/pr-merge.md ] && cat /tmp/pr-merge.md
echo
case "$BUILD_OUTCOME" in
success)
case "$BUILD_OUTCOME:$UNIT_TEST_OUTCOME:$APK_OUTCOME" in
success:success:success)
echo "### Build passed"
echo
echo "The \`droiddeck-apk\` artifact is [on this run]($RUN_URL), signed with the public test key."
@@ -568,7 +603,28 @@ jobs:
echo
echo "Fallback: [open **PR Release Publish**]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/workflows/pr-release-publish.yml) on \`main\` and enter PR \`$PR\`."
;;
failure)
success:success:*)
echo "### APK publication incomplete"
echo
echo "Compilation and unit tests passed, but APK publication did not complete. See [the run]($RUN_URL)."
;;
success:failure:*)
echo "### ERROR: Unit tests failed"
echo
echo "The APK compiled, but unit tests failed. No installable artifact was published."
echo
if [ -n "$TEST_DIAGNOSTICS_URL" ]; then
echo "The [unit-test-results]($TEST_DIAGNOSTICS_URL) artifact contains the test log and any generated reports."
else
echo "The test diagnostics upload did not complete. See [the test log]($RUN_URL)."
fi
;;
success:*)
echo "### Build incomplete"
echo
echo "The APK compiled, but unit tests did not complete successfully. See [the run]($RUN_URL)."
;;
failure:*)
echo "### ERROR: Build failed"
echo
echo "The compiler's errors ([full log]($RUN_URL)):"
@@ -586,7 +642,7 @@ jobs:
echo '```'
echo "</details>"
;;
skipped|cancelled)
skipped:*|cancelled:*)
echo "### Not built"
echo
echo "The build did not run (see above, or [the run]($RUN_URL))."
@@ -602,14 +658,6 @@ jobs:
gh api -X POST "repos/${GITHUB_REPOSITORY}/issues/${PR}/comments" -F body=@/tmp/pr-report.md >/dev/null
fi
- uses: actions/upload-artifact@v4
with:
name: droiddeck-apk
path: app/build/outputs/apk/release/*.apk
if-no-files-found: error
retention-days: 30
compression-level: 0
sign:
needs: build
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
+13 -1
View File
@@ -15,7 +15,7 @@ DroidDeck brings the SteamOS experience to Android: Valve's Steam client in Big
## Requirements and install
Use Android 9 or newer on a supported Adreno device (730 or newer, or 8xx). Adreno 6xx is experimental: DirectX 11 uses DXVK 2 and may run, and DirectX 12 games can still crash. Mali, Xclipse, PowerVR, and Adreno 710 are unsupported. No root is required. Allow about 3 GB for the runtime and 1.1 GB more for the desktop and emulators. Install the APK from [Releases](https://github.com/Droid-Deck/DroidDeck/releases), install the Linux runtime, then press **Play** and sign in. Steam downloads on first launch. Install **Desktop & apps** to use the desktop and emulators. The **Store** installs Linux apps and games from Flathub (ARM64 builds) with Flatpak; with additional options to install Appimages and set up scripts. **Stores** (Setup › Stores › "Show Stores in the rail") signs in to GOG, Epic Games and Amazon Games, browses their catalogs and libraries, downloads games into the Games storage and adds each one to the Steam client as a non-Steam game, so it launches through Proton like everything else; the cog on its chip row sets which tab a store opens on and the download speed tier.
Use Android 9 or newer on a supported Adreno device (730 or newer, or 8xx). Adreno 6xx is experimental: DirectX 11 uses DXVK 2 and may run, and DirectX 12 games can still crash. Mali, Xclipse, PowerVR, and Adreno 710 are unsupported. No root is required. Allow about 3 GB for the runtime and 1.1 GB more for the desktop and emulators. Install the APK from [Releases](https://github.com/Droid-Deck/DroidDeck/releases), install the Linux runtime, then press **Play** and sign in. Steam downloads on first launch. Install **Desktop & apps** to use the desktop and emulators. The **Store** installs Linux apps and games from Flathub (ARM64 builds) with Flatpak; with additional options to install Appimages and set up scripts. **Stores** (Setup › Stores › "Show Stores in the rail") signs in to GOG, Epic Games and Amazon Games and downloads games into the Games storage. Epic keeps its direct API install path. Opening Epic Stores prepares Epic’s official Linux launcher preview. When installed, Epic games launched from DroidDeck use that client, and games installed in the client are imported into DroidDeck’s library and shortcuts. The x86-64 client runs through FEX on ARM and DroidDeck retains its Proton hooks; this preview and ARM setup are outside Epic’s listed Linux support. The cog on a store’s chip row sets its opening tab and download speed tier.
On Android 14 and newer, before Steam launches, you must turn on **Disable child process restrictions** in Developer options. If this option is not available in developer settings (Android 12 and 13 devices), first launch of Steam will present a "Fix it for me" button, which will help automate the setup process.
@@ -29,6 +29,12 @@ Run `tools/build_local.sh` with Docker, Java 17, the Android SDK/NDK, Rust throu
After preparation, ordinary `./gradlew :app:assembleRelease` builds can reuse the verified inputs. If native sources, component pins or staged files change, packaging stops with instructions to rerun the helper. Generated audio stays under `app/build/`; the tracked base bundle is never modified. `./gradlew -PskipRust=true :app:testDebugUnitTest` needs no native preparation. For an APK without store engines, use `DROIDDECK_SKIP_RUST=1 tools/build_local.sh`; it omits cached engine libraries too.
For the prepared Windows checkout, run `./tools/run_windows.ps1` (optionally with `-Serial <adb-serial>`), or select **DroidDeck XR** in Android Studio and run the **app** module on the connected phone. Machine-specific options live in the ignored `local.properties`: `droiddeck.python` selects a Python executable, `droiddeck.skipRust=true` omits the optional store download engines, and `droiddeck.debugApplicationIdSuffix=.xr` installs **DroidDeck XR Dev** alongside the signed release. The development app has its own runtime and Steam login. The local Java selection is in `.gradle/config.properties`.
`python tools/prepare_windows_runtime.py` restores this checkout's audited runtime inputs from the pinned CI APK, three locally rebuilt preload libraries, and the checksum-pinned Gamescope archive in `build/windows-setup/`. Start Docker Desktop's Linux engine first: preparation now rebuilds DroidDeck's ARM64 VR driver and Vulkan layer from current source and runs their native regression tests. The compiler image is built automatically on first use. Vulkan headers come from the NDK beside `--llvm-strip`, or from an explicit `--vk-include <NDK-sysroot-include>` directory.
To reuse that build, pass `--rebuilt-vr build/windows-setup/vr`. Preparation verifies its `vr-build.json` against the current Git revision, source files, Vulkan headers, output checksums, and ARM64 architecture; missing or stale builds fail instead of falling back to the donor's VR libraries. The script checks the actual donor CI merge tree, still rejects unrelated unaudited runtime changes, and records component provenance in `app/build/windows-runtime-provenance.json`. Preserve these ignored preparation files when rebuilding locally; `clean` removes the prepared assets and their verification manifest.
Run the helper and preload regression suite with `bash tools/test_local.sh`. On macOS it runs in Linux through Docker, matching the helpers' runtime and CI instead of compiling Linux preloads against macOS headers.
## Limits
@@ -38,3 +44,9 @@ Compatibility and performance vary by device; hardware validation is limited. De
## Credits and licence
GPL-3.0. Runtime, shim, input, and controller work build on WinNative and Bannerlator (maxjivi05). Epic Games and GOG support build on [Phobos’s](https://github.com/phobos665) adaptation of [Legendary](https://github.com/legendary-gl/legendary) and [gogdl](https://github.com/Heroic-Games-Launcher/heroic-gogdl), respectively. LSFG frame generation is from the work of Camille LaVey and the [Eden](https://eden-emu.dev) emulator project, following [lsfg-vk](https://github.com/PancakeTAS/lsfg-vk), ported to WinNative and DroidDeck by [@maxjivi05](https://github.com/maxjivi05); it needs your own copy of [Lossless Scaling](https://store.steampowered.com/app/993090/) and ships none of its shaders. x86 AppImages, and any whose own runtime cannot unpack them, are unpacked with [uruntime](https://github.com/VHSgunzo/uruntime) by VHSgunzo (MIT), shipped unmodified with its licence. mmap ring-buffer controller input was developed by ribbit384 ([WinNative #345](https://github.com/WinNative-Emu/WinNative/pull/345)). See [LICENSE](LICENSE). Steam and Proton belong to Valve Corporation; this project is not affiliated with Valve.
XR glasses support builds on WinNative's VITURE glasses work. VITURE protocol details were checked against [XRLinuxDriver](https://github.com/wheaney/XRLinuxDriver) by [@wheaney](https://github.com/wheaney), including the VITURE protocol header it ships, [ar-drivers-rs](https://github.com/badicsalex/ar-drivers-rs) by [@badicsalex](https://github.com/badicsalex) and the [go-macos/iokit VITURE package](https://pkg.go.dev/github.com/go-macos/iokit/viture); the optional VITURE SDK belongs to VITURE and is not included. RayNeo support adapts [@informalTechCode](https://github.com/informalTechCode)'s [RayNeo-Air-Series-3DoF-Scaffold](https://github.com/informalTechCode/RayNeo-Air-Series-3DoF-Scaffold) (Apache-2.0) and references [@verncat](https://github.com/verncat)'s [RayNeo-Air-3S-Pro-OpenVR](https://github.com/verncat/RayNeo-Air-3S-Pro-OpenVR) (MIT) and [Monado](https://gitlab.freedesktop.org/monado/monado) (BSL-1.0); see [docs/third-party/rayneo](docs/third-party/rayneo/README.md). XREAL 1S support references [XRLinuxDriver](https://github.com/wheaney/XRLinuxDriver) and [xrealOneDeviceKit](https://github.com/wheaney/xrealOneDeviceKit) by @wheaney, [xreal_one_driver](https://github.com/rohitsangwan01/xreal_one_driver) by [@rohitsangwan01](https://github.com/rohitsangwan01) (MIT) and [xr-tools](https://github.com/0xcaff/xr-tools) by [@0xcaff](https://github.com/0xcaff). The REDMAGIC glasses-display findings draw on [@mekhontsev](https://github.com/mekhontsev)'s [MagicDesk investigation](https://github.com/mekhontsev/magicdesk/blob/main/docs/xr-resolution-fix.md) and [@Coding-BR](https://github.com/Coding-BR)'s [decompiled Nubia kernel](https://github.com/Coding-BR/android_kernel_nubia_sm8850_qwjujube). VITURE, RayNeo, XREAL and REDMAGIC are trademarks of their owners; this project is not affiliated with them.
Joy-Con VR controllers port the arm model of Google's [Google VR SDK for Unity](https://github.com/googlevr/gvr-unity-sdk) (Apache-2.0), adapt the hand skeleton of Valve's [OpenVR](https://github.com/ValveSoftware/openvr) `handskeletonsimulation` sample (BSD-3-Clause), which follows Moshi Turner's hand simulation in [Monado](https://gitlab.freedesktop.org/monado/monado), and use the Index controller offsets from Monado's `vive_poses.c` (BSL-1.0). Joy-Con behaviour was researched in Linux's [hid-nintendo](https://github.com/torvalds/linux/blob/master/drivers/hid/hid-nintendo.c) driver by Daniel J. Ogorchock, [@dekuNukem](https://github.com/dekuNukem)'s [Nintendo_Switch_Reverse_Engineering](https://github.com/dekuNukem/Nintendo_Switch_Reverse_Engineering), [SDL](https://github.com/libsdl-org/SDL) and the Android Open Source Project, and checked against [ALVR](https://github.com/alvr-org/ALVR), [OpenComposite](https://gitlab.com/znixian/OpenOVR), [PSMoveSteamVRBridge](https://github.com/HipsterSloth/PSMoveSteamVRBridge), [@F8LEFT](https://github.com/F8LEFT)'s [JoyConVr](https://github.com/F8LEFT/JoyConVr), [@krnelpan1c](https://github.com/krnelpan1c)'s [Daydream-Controller-SteamVR](https://github.com/krnelpan1c/Daydream-Controller-SteamVR), [@yoyobuae](https://github.com/yoyobuae)'s [joyconvr](https://github.com/yoyobuae/joyconvr), [Moonlight](https://github.com/moonlight-stream/moonlight-android) and [Dolphin](https://github.com/dolphin-emu/dolphin); see [docs/third-party/joycon](docs/third-party/joycon/README.md). Nintendo Switch and Joy-Con are trademarks of Nintendo; this project is not affiliated with Nintendo.
Camera hand tracking runs Google's [MediaPipe](https://github.com/google-ai-edge/mediapipe) Tasks Vision library with its Pose Landmarker (Lite) model, BlazePose GHUM 3D, and its Hand Landmarker model (all Apache-2.0), on frames from AndroidX [CameraX](https://developer.android.com/jetpack/androidx/releases/camera) (Apache-2.0); MediaPipe's usage reports are switched off. The design was checked against [@F8LEFT](https://github.com/F8LEFT)'s [JoyConVr](https://github.com/F8LEFT/JoyConVr) camera app (MIT), and the short Joy-Con movements follow the accelerometer mode of the [Google VR SDK for Unity](https://github.com/googlevr/gvr-unity-sdk)'s arm model, Eric Foxlin's zero-velocity updates and Frosio, Pedersini and Borghese's accelerometer autocalibration, and the camera and Joy-Con are joined by a Kalman filter (R. E. Kalman, 1960). Bare camera hands take their gestures after [ALVR](https://github.com/alvr-org/ALVR)'s `hand_gestures.rs` (MIT) and Meta Quest's menu pinch, with distances checked on photos from the [HaGRID](https://github.com/hukenovs/hagrid) gesture set (CC BY-SA 4.0, not included), fit the palm with Horn's quaternion method (1987) and smooth it with the 1€ filter (Casiez, Roussel and Vogel, 2012); the white glove model follows Valve's OpenVR render model reference and Index controller poses, is meshed with surface nets (Gibson, 1998; Mikola Lysenko) and blends and shades it with Inigo Quilez's smooth minimum and distance-field ambient occlusion, porting his polygon distance (MIT); see [docs/third-party/handtracking](docs/third-party/handtracking/README.md).
+48 -2
View File
@@ -4,6 +4,25 @@ plugins {
apply plugin: 'kotlin-android'
apply plugin: 'org.jetbrains.kotlin.plugin.compose'
// Machine-specific tooling and optional CI-prepared inputs stay out of version control.
def localBuildOptions = new Properties()
def localBuildOptionsFile = rootProject.layout.projectDirectory.file('local.properties')
if (localBuildOptionsFile.asFile.exists()) {
localBuildOptions.load(new StringReader(providers.fileContents(localBuildOptionsFile).asText.get()))
}
def pythonExecutable = project.findProperty('pythonExecutable')
?: localBuildOptions.getProperty('droiddeck.python', 'python3')
def vitureSdkDir = project.findProperty('vitureSdkDir') ?: localBuildOptions.getProperty('droiddeck.vitureSdk', '')
if (vitureSdkDir) {
['android/arm64-v8a/libglasses.so', 'android/arm64-v8a/libcarina_vio.so',
'android/arm64-v8a/libcloud_protocol.so', 'include/viture_glasses_provider.h',
'android/LICENSE', 'android/NOTICE'].each { required ->
if (!new File(vitureSdkDir, required).isFile()) {
throw new GradleException("Incomplete VITURE Android SDK: missing ${required}")
}
}
}
def gitValue = { List<String> arguments ->
try {
def output = providers.exec {
@@ -100,6 +119,7 @@ android {
externalNativeBuild {
cmake {
arguments '-DANDROID_STL=c++_shared'
arguments "-DDROIDDECK_VITURE_SDK_DIR=${vitureSdkDir.toString().replace('\\', '/')}"
}
}
}
@@ -128,6 +148,11 @@ android {
debug {
minifyEnabled false
signingConfig signingConfigs.testkey
def localSuffix = localBuildOptions.getProperty('droiddeck.debugApplicationIdSuffix')
if (localSuffix) {
applicationIdSuffix localSuffix
resValue 'string', 'app_name', 'DroidDeck XR Dev'
}
}
release {
minifyEnabled false
@@ -158,6 +183,10 @@ android {
testOptions {
unitTests.returnDefaultValues = true
unitTests.includeAndroidResources = true
unitTests.all {
def logTarget = providers.systemProperty('robolectric.logging').orNull
if (logTarget != null) systemProperty 'robolectric.logging', logTarget
}
}
compileOptions {
@@ -169,6 +198,16 @@ android {
}
}
// Vendor binaries remain in the user's SDK directory; never vendor them into Git.
if (vitureSdkDir) {
android.sourceSets.main.jniLibs.srcDir(new File(vitureSdkDir, 'android'))
def stageVitureNotices = tasks.register('stageVitureNotices', Copy) {
from(new File(vitureSdkDir, 'android')) { include 'LICENSE', 'NOTICE' }
into(layout.buildDirectory.dir('vitureAssets/licenses/viture'))
}
tasks.named('preBuild').configure { dependsOn(stageVitureNotices) }
}
def stageSessionScripts = tasks.register('stageSessionScripts', Copy) {
from(rootProject.file('tools/linuxfs/overlay')) {
include 'usr/local/bin/droiddeck-*', 'usr/local/bin/steam-compatibility', 'usr/bin/**'
@@ -185,12 +224,13 @@ def stageRuntimeAssets = tasks.register('stageRuntimeAssets', Sync) {
into(runtimeAssetsDir)
}
android.sourceSets.main.assets.setSrcDirs([runtimeAssetsDir])
if (vitureSdkDir) android.sourceSets.main.assets.srcDir(layout.buildDirectory.dir('vitureAssets'))
tasks.named('preBuild').configure { dependsOn(stageRuntimeAssets) }
def verifyRuntimeInputs = tasks.register('verifyRuntimeInputs', Exec) {
mustRunAfter(stageSessionScripts, stageRuntimeAssets)
workingDir rootProject.projectDir
commandLine 'python3', 'tools/release/runtime_inputs.py', 'verify'
commandLine pythonExecutable, 'tools/release/runtime_inputs.py', 'verify'
}
tasks.configureEach { task ->
if (task.name in ['packageDebug', 'packageRelease']) task.dependsOn(verifyRuntimeInputs)
@@ -198,7 +238,8 @@ tasks.configureEach { task ->
def rustCrateDir = layout.projectDirectory.dir('src/main/rust/stores')
def rustJniLibsDir = layout.buildDirectory.dir('rustJniLibs')
def skipRust = (project.findProperty('skipRust') ?: 'false').toString().toBoolean()
def skipRust = (project.findProperty('skipRust')
?: localBuildOptions.getProperty('droiddeck.skipRust', 'false')).toString().toBoolean()
def buildRustStores = tasks.register('buildRustStores', Exec) {
description = 'Cross-compiles the store download engines (libdroiddeckstores.so) with cargo-ndk.'
onlyIf { !skipRust }
@@ -242,6 +283,11 @@ dependencies {
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3'
implementation 'com.flyfishxu:kadb-android:1.2.1'
implementation 'com.termux.termux-app:terminal-view:0.118.0'
implementation 'androidx.camera:camera-camera2:1.4.2'
implementation 'androidx.camera:camera-lifecycle:1.4.2'
implementation('com.google.mediapipe:tasks-vision:0.10.21') {
exclude group: 'com.google.auto.value', module: 'auto-value'
}
testImplementation 'junit:junit:4.13.2'
testImplementation 'org.robolectric:robolectric:4.16.1'
+24 -1
View File
@@ -1,5 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CHANGE_WIFI_MULTICAST_STATE" />
@@ -19,14 +20,19 @@
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" />
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.vulkan.version" android:version="0x401000" android:required="true" />
<uses-feature android:name="android.hardware.gamepad" android:required="false" />
<uses-feature android:name="android.hardware.camera" android:required="false" />
<uses-feature android:name="android.hardware.camera.front" android:required="false" />
<uses-feature android:name="android.hardware.camera.autofocus" android:required="false" />
<uses-feature android:name="android.hardware.vr.headtracking" android:version="1" android:required="false" />
<application
android:name=".App"
android:localeConfig="@xml/locales_config"
android:networkSecurityConfig="@xml/network_security_config"
android:allowBackup="false"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
@@ -145,6 +151,16 @@
android:resizeableActivity="true"
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|keyboard|keyboardHidden|navigation|uiMode|density" />
<activity
android:name=".gpu.SteamVrRestartActivity"
android:exported="false"
android:process=":vr_recovery"
android:taskAffinity="${applicationId}.vr_recovery"
android:excludeFromRecents="true"
android:screenOrientation="sensorLandscape"
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|keyboard|keyboardHidden|navigation|uiMode|density"
android:theme="@style/SessionTheme" />
<service
android:name=".session.SessionService"
android:exported="false"
@@ -184,5 +200,12 @@
android:exported="true"
android:permission="android.permission.DUMP"
android:grantUriPermissions="false" />
<service
android:name="com.google.android.datatransport.runtime.backends.TransportBackendDiscovery"
tools:node="merge">
<meta-data
android:name="backend:com.google.android.datatransport.cct.CctBackendFactory"
tools:node="remove" />
</service>
</application>
</manifest>
+9
View File
@@ -38,3 +38,12 @@ add_library(deviceinfo SHARED deviceinfo/vkinfo.c)
target_link_libraries(deviceinfo dl)
add_library(termux SHARED termux/termux.c)
# The optional proprietary SDK is supplied locally; its headers/libraries are not vendored.
if(DROIDDECK_VITURE_SDK_DIR)
add_library(droiddeck_viture SHARED viture/sdk_bridge.cpp)
target_compile_features(droiddeck_viture PRIVATE cxx_std_17)
target_include_directories(droiddeck_viture PRIVATE "${DROIDDECK_VITURE_SDK_DIR}/include")
target_link_directories(droiddeck_viture PRIVATE "${DROIDDECK_VITURE_SDK_DIR}/android/${ANDROID_ABI}")
target_link_libraries(droiddeck_viture PRIVATE glasses log)
endif()
+36
View File
@@ -0,0 +1,36 @@
#pragma once
// Our controller uses the standard display-mode vocabulary, even for Beast.
// Native mode reuses some numbers with different meanings; never pass them through.
namespace droiddeck::viture {
constexpr int to_native_mode(int standard) {
switch (standard) {
case 0x31: return 0x31; // 1080p 60
case 0x33: return 0x32; // 1080p 90
case 0x34: return 0x33; // 1080p 120
case 0x32: return 0x37; // 1080p SBS 60
case 0x35: return 0x38; // 1080p SBS 90
case 0x41: return 0x34; // 1200p 60
case 0x43: return 0x35; // 1200p 90
case 0x44: return 0x36; // 1200p 120
case 0x42: return 0x3a; // 1200p SBS 60
case 0x45: return 0x3b; // 1200p SBS 90
default: return -4;
}
}
constexpr int from_native_mode(int native) {
switch (native) {
case 0x31: return 0x31;
case 0x32: return 0x33;
case 0x33: return 0x34;
case 0x37: return 0x32;
case 0x38: return 0x35;
case 0x34: return 0x41;
case 0x35: return 0x43;
case 0x36: return 0x44;
case 0x3a: return 0x42;
case 0x3b: return 0x45;
default: return -4; // Ultrawide is not stereo.
}
}
}
+308
View File
@@ -0,0 +1,308 @@
#include <jni.h>
#include <android/log.h>
#include <algorithm>
#include <array>
#include <cstdint>
#include <memory>
#include <mutex>
#include <time.h>
#include "viture_glasses_provider.h"
#include "viture_device.h"
#include "viture_device_carina.h"
#include "viture_protocol_public.h"
#include "viture_result.h"
#include "viture_version.h"
#include "display_modes.h"
namespace {
// SDK callbacks have no user-data parameter. Only one provider may exist at a time.
std::mutex api_mutex;
std::mutex pose_mutex;
std::array<float, 4> latest_pose{};
int64_t latest_time = 0;
bool accepting_pose = false;
bool accepting_state = false;
std::array<int, 6> state_values{};
unsigned state_pending = 0;
void on_state(int id, int value) {
std::lock_guard<std::mutex> lock(pose_mutex);
if (!accepting_state || id < 0 || id >= int(state_values.size())) return;
state_values[id] = value;
state_pending |= 1u << id;
}
void clear_state(bool accept) {
std::lock_guard<std::mutex> lock(pose_mutex);
accepting_state = accept;
state_pending = 0;
}
int64_t now_ns() {
timespec ts{};
clock_gettime(CLOCK_BOOTTIME, &ts);
return int64_t(ts.tv_sec) * 1000000000LL + ts.tv_nsec;
}
void on_pose(float* data, uint64_t) {
if (!data) return;
std::lock_guard<std::mutex> lock(pose_mutex);
if (!accepting_pose) return;
std::copy_n(data + 3, 4, latest_pose.begin());
// SDK device ticks are not Android's elapsed-realtime clock.
latest_time = now_ns();
}
void clear_pose(bool accept) {
std::lock_guard<std::mutex> lock(pose_mutex);
accepting_pose = accept;
latest_time = 0;
}
struct Session {
XRDeviceProviderHandle device = nullptr;
int pid = 0;
int type = -1;
bool initialized = false;
bool started = false;
bool tracking = false;
bool native_dof = false;
bool switched_native = false;
int original_bypass_display = -1;
int saved_dof = -1;
int64_t consumed_time = 0;
jlong token = 0;
};
std::unique_ptr<Session> session;
jlong next_token = 0;
void report(const char* step, int rc) {
if (rc < 0) __android_log_print(ANDROID_LOG_WARN, "VitureSdk", "%s: %d", step, rc);
}
int suspend_native_dof(Session& s) {
if (!s.native_dof) return 0;
int mode = xr_device_provider_native_get_mode(s.device);
if (mode < 0) return mode;
if (mode != 1) return 0;
if (s.saved_dof < 0) {
int dof = xr_device_provider_native_get_dof(s.device);
if (dof < 0) return dof;
s.saved_dof = dof;
}
return xr_device_provider_native_set_dof(s.device, VITURE_NATIVE_DOF_0);
}
void restore_native_dof(Session& s) {
if (s.saved_dof >= 0 && xr_device_provider_native_get_mode(s.device) == 1) {
report("restore native DOF", xr_device_provider_native_set_dof(s.device, s.saved_dof));
}
s.saved_dof = -1;
}
int tracking(Session& s, bool enabled) {
if (s.tracking == enabled) return 0;
if (!enabled) {
clear_pose(false);
int rc = s.type == XR_DEVICE_TYPE_VITURE_CARINA ? 0 :
xr_device_provider_close_imu(s.device, VITURE_IMU_MODE_POSE);
s.tracking = false;
restore_native_dof(s);
return rc;
}
int rc = suspend_native_dof(s);
if (rc < 0) return rc;
clear_pose(true);
s.consumed_time = 0;
if (s.type != XR_DEVICE_TYPE_VITURE_CARINA) {
int frequency = VITURE_IMU_FREQUENCY_MEDIUM_HIGH;
while (frequency >= VITURE_IMU_FREQUENCY_LOW &&
!xr_device_provider_is_product_support_imu_frequency(s.pid, VITURE_IMU_MODE_POSE, frequency)) --frequency;
rc = frequency < 0 ? VITURE_GLASSES_ERROR_NOT_SUPPORTED :
xr_device_provider_open_imu(s.device, VITURE_IMU_MODE_POSE, frequency);
}
if (rc == 0) s.tracking = true;
else { clear_pose(false); restore_native_dof(s); }
return rc;
}
void destroy(Session& s) {
clear_pose(false);
clear_state(false);
if (s.device) {
if (s.tracking) report("close tracking", tracking(s, false));
if (s.switched_native && s.original_bypass_display >= 0) {
int rc = xr_device_provider_native_set_mode(s.device, 0);
if (rc == 0) rc = xr_device_provider_set_display_mode(s.device, s.original_bypass_display);
report("restore bypass mode", rc);
}
if (s.started) report("stop", xr_device_provider_stop(s.device));
if (s.initialized) report("shutdown", xr_device_provider_shutdown(s.device));
xr_device_provider_destroy(s.device);
s.device = nullptr;
}
}
int get_display(Session& s) {
if (s.native_dof) {
const int mode = xr_device_provider_native_get_mode(s.device);
if (mode < 0) return mode;
if (mode == 1) {
const int value = xr_device_provider_native_get_display_mode(s.device);
return value < 0 ? value : droiddeck::viture::from_native_mode(value);
}
}
return xr_device_provider_get_display_mode(s.device);
}
int set_display(Session& s, int standard) {
if (!s.native_dof) return xr_device_provider_set_display_mode(s.device, standard);
int mode = xr_device_provider_native_get_mode(s.device);
if (mode < 0) return mode;
if (mode == 0) {
int rc = xr_device_provider_set_display_mode(s.device, standard);
if (rc != VITURE_GLASSES_ERROR_NOT_SUPPORTED) return rc;
// Older Beast firmware exposes fewer bypass modes. Fall back to its
// separate native table only when the SDK explicitly rejects the mode.
}
const int native = droiddeck::viture::to_native_mode(standard);
if (native < 0) return native;
if (mode == 0) {
if (!s.switched_native) {
s.original_bypass_display = xr_device_provider_get_display_mode(s.device);
if (s.original_bypass_display < 0) return s.original_bypass_display;
}
int rc = xr_device_provider_native_set_mode(s.device, 1);
if (rc < 0) return rc;
s.switched_native = true;
}
int rc = xr_device_provider_native_set_display_mode(s.device, native);
if (rc == 0 && s.tracking) rc = suspend_native_dof(s);
if (rc < 0 && mode == 0) {
restore_native_dof(s);
int rollback = xr_device_provider_native_set_mode(s.device, 0);
if (rollback == 0) rollback = xr_device_provider_set_display_mode(s.device, s.original_bypass_display);
report("rollback display mode", rollback);
if (rollback == 0) s.switched_native = false;
}
return rc;
}
}
#define JNI_METHOD(name) Java_com_droiddeck_launcher_glasses_VitureSdk_##name
extern "C" JNIEXPORT jboolean JNICALL JNI_METHOD(nativeAvailable)(JNIEnv*, jobject) {
__android_log_print(ANDROID_LOG_INFO, "VitureSdk", "SDK %s", GetVersionString());
return JNI_TRUE;
}
extern "C" JNIEXPORT jstring JNICALL JNI_METHOD(nativeMarketName)(JNIEnv* env, jobject, jint pid) {
std::lock_guard<std::mutex> lock(api_mutex);
if (!xr_device_provider_is_product_id_valid(pid)) return nullptr;
char name[64]{};
int length = sizeof(name) - 1;
if (xr_device_provider_get_market_name(pid, name, &length) != 0) return nullptr;
name[sizeof(name) - 1] = 0;
return env->NewStringUTF(name);
}
extern "C" JNIEXPORT jlong JNICALL JNI_METHOD(nativeOpen)(JNIEnv* env, jobject, jint pid, jint fd, jstring cache) {
std::lock_guard<std::mutex> lock(api_mutex);
if (session || fd < 0 || !cache || !xr_device_provider_is_product_id_valid(pid)) return 0;
auto s = std::make_unique<Session>();
s->pid = pid;
s->device = xr_device_provider_create(pid, fd);
if (!s->device) return 0;
s->type = xr_device_provider_get_device_type(s->device);
s->native_dof = xr_device_provider_is_product_support_native_dof(pid) == 1;
int rc = s->type < 0 ? s->type : 0;
if (rc == 0) {
clear_state(true);
rc = xr_device_provider_register_state_callback(s->device, on_state);
}
if (rc == 0) {
rc = s->type == XR_DEVICE_TYPE_VITURE_CARINA ?
xr_device_provider_set_dof_type_carina(s->device, 0) :
xr_device_provider_register_imu_pose_callback(s->device, on_pose);
}
if (rc == 0) {
const char* directory = env->GetStringUTFChars(cache, nullptr);
if (!directory) { destroy(*s); return 0; }
rc = xr_device_provider_initialize(s->device, nullptr, directory);
env->ReleaseStringUTFChars(cache, directory);
s->initialized = rc == 0;
}
if (rc == 0) {
rc = xr_device_provider_start(s->device);
s->started = rc == 0;
}
if (rc != 0) { report("initialize/start", rc); destroy(*s); return 0; }
s->token = ++next_token;
session = std::move(s);
return session->token;
}
extern "C" JNIEXPORT jint JNICALL JNI_METHOD(nativeType)(JNIEnv*, jobject, jlong token) {
std::lock_guard<std::mutex> lock(api_mutex);
return session && session->token == token ? session->type : -1;
}
extern "C" JNIEXPORT jint JNICALL JNI_METHOD(nativeTracking)(JNIEnv*, jobject, jlong token, jboolean enabled) {
std::lock_guard<std::mutex> lock(api_mutex);
return session && session->token == token ? tracking(*session, enabled) : -1;
}
extern "C" JNIEXPORT jint JNICALL JNI_METHOD(nativeCommand)(JNIEnv*, jobject, jlong token, jint command, jint value) {
std::lock_guard<std::mutex> lock(api_mutex);
if (!session || session->token != token) return -1;
auto& s = *session;
switch (command) {
case 1: return get_display(s);
case 2: return set_display(s, value);
case 3: return xr_device_provider_get_brightness_level(s.device);
case 4: return xr_device_provider_set_brightness_level(s.device, value);
case 5: return xr_device_provider_get_volume_level(s.device);
case 6: return xr_device_provider_set_volume_level(s.device, value);
case 7: return xr_device_provider_set_film_mode(s.device, value ? 1.0f : 0.0f);
default: return VITURE_GLASSES_ERROR_NOT_SUPPORTED;
}
}
extern "C" JNIEXPORT jlong JNICALL JNI_METHOD(nativePose)(JNIEnv* env, jobject, jlong token, jfloatArray output) {
std::lock_guard<std::mutex> lock(api_mutex);
if (!session || session->token != token || !output || env->GetArrayLength(output) != 4) return -1;
auto& s = *session;
if (!s.tracking) return 0;
if (s.type == XR_DEVICE_TYPE_VITURE_CARINA) {
float pose[7]{};
int status = 1;
int rc = xr_device_provider_get_gl_pose_carina(s.device, pose, 0.0, &status);
if (rc < 0) return rc;
if (status != 0) return 0;
env->SetFloatArrayRegion(output, 0, 4, pose + 3);
return now_ns();
}
std::lock_guard<std::mutex> pose_lock(pose_mutex);
if (!latest_time || latest_time == s.consumed_time) return 0;
env->SetFloatArrayRegion(output, 0, 4, latest_pose.data());
s.consumed_time = latest_time;
return latest_time;
}
extern "C" JNIEXPORT jint JNICALL JNI_METHOD(nativeNextState)(JNIEnv* env, jobject, jlong token, jintArray output) {
std::lock_guard<std::mutex> lock(api_mutex);
if (!session || session->token != token || !output || env->GetArrayLength(output) != 2) return -1;
std::lock_guard<std::mutex> state_lock(pose_mutex);
for (int id = 0; id < int(state_values.size()); ++id) {
if (!(state_pending & (1u << id))) continue;
state_pending &= ~(1u << id);
const jint value[] = {id, state_values[id]};
env->SetIntArrayRegion(output, 0, 2, value);
return 1;
}
return 0;
}
extern "C" JNIEXPORT void JNICALL JNI_METHOD(nativeClose)(JNIEnv*, jobject, jlong token) {
std::lock_guard<std::mutex> lock(api_mutex);
if (session && session->token == token) { destroy(*session); session.reset(); }
}
+27 -10
View File
@@ -34,16 +34,19 @@ void cardboard_clamp(struct cardboard_params *p) {
p->screen_width_m = clampf(p->screen_width_m, 0.2f, 60.0f);
p->k1 = clampf(p->k1, -1.0f, 2.0f);
p->k2 = clampf(p->k2, -1.0f, 2.0f);
p->fov_deg = clampf(p->fov_deg, 30.0f, 140.0f);
// Optical vertical FOV on supported glasses can be as low as ~22 degrees.
p->fov_deg = clampf(p->fov_deg, 20.0f, 140.0f);
p->lens_inset = clampf(p->lens_inset, -0.25f, 0.25f);
p->chroma = clampf(p->chroma, 0.0f, 0.05f);
p->mono = p->mono ? 1 : 0;
p->display_hz = clampf(p->display_hz, 0.0f, 240.0f);
}
void cardboard_push_constants(const struct cardboard_params *p, const float pose[9],
int src_w, int src_h, int out_w, int out_h, float out[24]) {
for (int r = 0; r < 3; r++)
for (int c = 0; c < 3; c++) out[r * 4 + c] = pose[r * 3 + c];
out[3] = p->ipd_m * 0.5f;
out[3] = p->mono ? 0.0f : p->ipd_m * 0.5f;
out[7] = p->screen_dist_m;
out[11] = p->screen_width_m * 0.5f;
out[12] = p->k1;
@@ -54,8 +57,9 @@ void cardboard_push_constants(const struct cardboard_params *p, const float pose
out[17] = (src_w > 0 && src_h > 0) ? (float)src_w / (float)src_h : 16.0f / 9.0f;
if (p->source == CARDBOARD_SOURCE_SBS) out[17] *= 0.5f;
out[18] = (float)p->source;
out[19] = (out_w > 0 && out_h > 0) ? ((float)out_w * 0.5f) / (float)out_h : 1.0f;
out[20] = out[21] = out[22] = out[23] = 0.0f;
out[19] = (out_w > 0 && out_h > 0) ? ((float)out_w * (p->mono ? 1.0f : 0.5f)) / (float)out_h : 1.0f;
out[20] = p->mono ? 1.0f : 0.0f;
out[21] = out[22] = out[23] = 0.0f;
}
#ifndef CARDBOARD_HOST_TEST
@@ -81,13 +85,21 @@ void vkp_cardboard_set(const struct cardboard_params *p) {
shm_publish(g_live);
pthread_mutex_unlock(&g_lock);
if (was != c.enabled || c.enabled)
droiddeck_log("cardboard", "%s: %s, ipd %.0f mm, screen %.1f m wide at %.1f m, lens k1 %.2f k2 %.2f, fov %.0f, inset %.3f, chroma %.3f",
droiddeck_log("cardboard", "%s: %s%s, ipd %.0f mm, screen %.1f m wide at %.1f m, lens k1 %.2f k2 %.2f, fov %.0f, inset %.3f, chroma %.3f",
c.enabled ? "on" : "off", c.source == CARDBOARD_SOURCE_SBS ? "side-by-side source" : "virtual screen",
c.ipd_m * 1000.0f, c.screen_width_m, c.screen_dist_m, c.k1, c.k2, c.fov_deg, c.lens_inset, c.chroma);
c.mono ? " (one eye)" : "", c.ipd_m * 1000.0f, c.screen_width_m, c.screen_dist_m, c.k1, c.k2, c.fov_deg, c.lens_inset, c.chroma);
}
static struct droiddeck_headpose *g_shm;
static int g_eye_w, g_eye_h;
static int g_render_height;
void vkp_cardboard_set_render_height(int height) {
pthread_mutex_lock(&g_lock);
g_render_height = height > 0 ? height : 0;
shm_publish(g_live);
pthread_mutex_unlock(&g_lock);
}
void vkp_cardboard_set_pose_file(const char *path) {
pthread_mutex_lock(&g_lock);
@@ -101,6 +113,7 @@ void vkp_cardboard_set_pose_file(const char *path) {
g_shm->magic = DROIDDECK_HEADPOSE_MAGIC;
g_shm->version = DROIDDECK_HEADPOSE_VERSION;
memcpy(g_shm->rot, g_pose, sizeof(g_shm->rot));
shm_publish(g_live);
}
}
if (fd >= 0) close(fd);
@@ -119,7 +132,10 @@ static void shm_publish(int live) {
atomic_thread_fence(memory_order_release);
memcpy(g_shm->rot, g_pose, sizeof(g_shm->rot));
g_shm->ipd_m = g_set.ipd_m;
g_shm->flags = (live ? DROIDDECK_HEADPOSE_LIVE : 0u) | (g_set.enabled ? DROIDDECK_HEADPOSE_ENABLED : 0u);
g_shm->flags = (live ? DROIDDECK_HEADPOSE_LIVE : 0u) | (g_set.enabled ? DROIDDECK_HEADPOSE_ENABLED : 0u) |
(g_set.mono ? DROIDDECK_HEADPOSE_MONO : 0u);
g_shm->display_hz = g_set.display_hz;
g_shm->render_height = (uint32_t)g_render_height;
g_shm->time_ns = (int64_t)ts.tv_sec * 1000000000LL + ts.tv_nsec;
g_shm->tan_half_fov = tanf(g_set.fov_deg * 0.5f * (float)M_PI / 180.0f);
g_shm->eye_w = (uint32_t)(g_eye_w > 0 ? g_eye_w : 0);
@@ -401,14 +417,15 @@ VkImage vkp_cardboard_run(VkCommandBuffer cmd, VkImage src, int src_w, int src_h
if (!said) { said = 1; droiddeck_log("error", "cardboard: no descriptor for the frame; shown flat"); }
return VK_NULL_HANDLE;
}
if (g_eye_w != out_w / 2 || g_eye_h != out_h) {
snapshot();
int eye_w = g_cur.mono ? out_w : out_w / 2;
if (g_eye_w != eye_w || g_eye_h != out_h) {
pthread_mutex_lock(&g_lock);
g_eye_w = out_w / 2;
g_eye_w = eye_w;
g_eye_h = out_h;
shm_publish(g_live);
pthread_mutex_unlock(&g_lock);
}
snapshot();
struct { float ndc[4]; float rest[24]; } pc = {{-1.0f, -1.0f, 1.0f, 1.0f}, {0}};
cardboard_push_constants(&g_cur, g_cur_pose, src_w, src_h, out_w, out_h, pc.rest);
@@ -10,7 +10,7 @@ layout(push_constant) uniform PC {
vec4 rot2;
vec4 lens;
vec4 view;
vec4 extra;
} pc;
const vec3 BACKDROP = vec3(0.015, 0.015, 0.02);
@@ -40,9 +40,10 @@ vec3 eyeSample(vec2 tan, float eye) {
}
void main() {
float eye = uv.x < 0.5 ? 0.0 : 1.0;
vec2 local = vec2(uv.x * 2.0 - eye, uv.y);
float cx = 0.5 + (eye < 0.5 ? pc.view.x : -pc.view.x);
bool mono = pc.extra.x > 0.5;
float eye = mono || uv.x < 0.5 ? 0.0 : 1.0;
vec2 local = mono ? uv : vec2(uv.x * 2.0 - eye, uv.y);
float cx = mono ? 0.5 : 0.5 + (eye < 0.5 ? pc.view.x : -pc.view.x);
vec2 p = local - vec2(cx, 0.5);
float tanH = pc.lens.z;
vec2 t = vec2(p.x * pc.view.w, p.y) * 2.0 * tanH;
@@ -15,11 +15,14 @@ struct cardboard_params {
float fov_deg;
float lens_inset;
float chroma;
int mono;
float display_hz;
};
void vkp_cardboard_set(const struct cardboard_params *p);
void vkp_cardboard_set_pose(const float m[9], int live);
void vkp_cardboard_set_pose_file(const char *path);
void vkp_cardboard_set_render_height(int height);
void vkp_cardboard_bind_device(VkDevice dev, const VkPhysicalDeviceMemoryProperties *memprops);
int vkp_cardboard_active(void);
+271 -253
View File
@@ -1,8 +1,8 @@
#pragma once
const uint32_t cardboard_frag_code[] = {
0x07230203,0x00010000,0x0008000b,0x0000016d,0x00000000,0x00020011,0x00000001,0x0006000b,
0x07230203,0x00010000,0x0008000b,0x00000185,0x00000000,0x00020011,0x00000001,0x0006000b,
0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,0x00000000,0x00000001,
0x0007000f,0x00000004,0x00000004,0x6e69616d,0x00000000,0x000000ee,0x00000165,0x00030010,
0x0007000f,0x00000004,0x00000004,0x6e69616d,0x00000000,0x000000f8,0x0000017d,0x00030010,
0x00000004,0x00000007,0x00030003,0x00000002,0x000001c2,0x00040005,0x00000004,0x6e69616d,
0x00000000,0x00070005,0x0000000d,0x69646e75,0x726f7473,0x66762874,0x31663b32,0x0000003b,
0x00030005,0x0000000b,0x00000074,0x00040005,0x0000000c,0x6c616373,0x00000065,0x00070005,
@@ -11,255 +11,273 @@ const uint32_t cardboard_frag_code[] = {
0x0000001e,0x00004350,0x00040006,0x0000001e,0x00000000,0x0063646e,0x00050006,0x0000001e,
0x00000001,0x30746f72,0x00000000,0x00050006,0x0000001e,0x00000002,0x31746f72,0x00000000,
0x00050006,0x0000001e,0x00000003,0x32746f72,0x00000000,0x00050006,0x0000001e,0x00000004,
0x736e656c,0x00000000,0x00050006,0x0000001e,0x00000005,0x77656976,0x00000000,0x00030005,
0x00000020,0x00006370,0x00040005,0x0000003f,0x486e6174,0x00000000,0x00030005,0x00000042,
0x00000073,0x00040005,0x00000068,0x72756f53,0x00006563,0x00030005,0x00000077,0x00000052,
0x00030005,0x00000094,0x00000064,0x00030005,0x0000009f,0x0000006f,0x00030005,0x000000b3,
0x00000074,0x00030005,0x000000bd,0x00746968,0x00040005,0x000000c3,0x666c6168,0x00000057,
0x00040005,0x000000c6,0x666c6168,0x00000048,0x00030005,0x000000cb,0x00000073,0x00030005,
0x000000ec,0x00657965,0x00030005,0x000000ee,0x00007675,0x00040005,0x000000f4,0x61636f6c,
0x0000006c,0x00030005,0x000000fe,0x00007863,0x00030005,0x0000010c,0x00000070,0x00040005,
0x00000111,0x486e6174,0x00000000,0x00030005,0x00000114,0x00000074,0x00030005,0x00000125,
0x00000063,0x00040005,0x00000129,0x61726170,0x0000006d,0x00040005,0x0000012b,0x61726170,
0x0000006d,0x00040005,0x0000012d,0x61726170,0x0000006d,0x00040005,0x0000012e,0x61726170,
0x0000006d,0x00040005,0x00000133,0x61726170,0x0000006d,0x00040005,0x00000135,0x61726170,
0x0000006d,0x00040005,0x00000137,0x61726170,0x0000006d,0x00040005,0x00000138,0x61726170,
0x0000006d,0x00040005,0x00000140,0x61726170,0x0000006d,0x00040005,0x00000142,0x61726170,
0x0000006d,0x00040005,0x00000144,0x61726170,0x0000006d,0x00040005,0x00000145,0x61726170,
0x0000006d,0x00040005,0x0000014b,0x61726170,0x0000006d,0x00040005,0x0000014d,0x61726170,
0x0000006d,0x00040005,0x0000014f,0x61726170,0x0000006d,0x00040005,0x00000150,0x61726170,
0x0000006d,0x00030005,0x00000153,0x00000065,0x00040005,0x00000159,0x65646166,0x00000000,
0x00050005,0x00000165,0x4374756f,0x726f6c6f,0x00000000,0x00030047,0x0000001e,0x00000002,
0x00050048,0x0000001e,0x00000000,0x00000023,0x00000000,0x00050048,0x0000001e,0x00000001,
0x00000023,0x00000010,0x00050048,0x0000001e,0x00000002,0x00000023,0x00000020,0x00050048,
0x0000001e,0x00000003,0x00000023,0x00000030,0x00050048,0x0000001e,0x00000004,0x00000023,
0x00000040,0x00050048,0x0000001e,0x00000005,0x00000023,0x00000050,0x00040047,0x00000068,
0x00000021,0x00000000,0x00040047,0x00000068,0x00000022,0x00000000,0x00040047,0x000000ee,
0x0000001e,0x00000000,0x00040047,0x00000165,0x0000001e,0x00000000,0x00020013,0x00000002,
0x00030021,0x00000003,0x00000002,0x00030016,0x00000006,0x00000020,0x00040017,0x00000007,
0x00000006,0x00000002,0x00040020,0x00000008,0x00000007,0x00000007,0x00040020,0x00000009,
0x00000007,0x00000006,0x00050021,0x0000000a,0x00000007,0x00000008,0x00000009,0x00040017,
0x0000000f,0x00000006,0x00000003,0x00050021,0x00000010,0x0000000f,0x00000008,0x00000009,
0x0004002b,0x00000006,0x0000001c,0x3f800000,0x00040017,0x0000001d,0x00000006,0x00000004,
0x0008001e,0x0000001e,0x0000001d,0x0000001d,0x0000001d,0x0000001d,0x0000001d,0x0000001d,
0x00040020,0x0000001f,0x00000009,0x0000001e,0x0004003b,0x0000001f,0x00000020,0x00000009,
0x00040015,0x00000021,0x00000020,0x00000001,0x0004002b,0x00000021,0x00000022,0x00000004,
0x00040015,0x00000023,0x00000020,0x00000000,0x0004002b,0x00000023,0x00000024,0x00000000,
0x00040020,0x00000025,0x00000009,0x00000006,0x0004002b,0x00000023,0x0000002b,0x00000001,
0x0004002b,0x00000021,0x00000036,0x00000005,0x0004002b,0x00000023,0x00000037,0x00000002,
0x0004002b,0x00000006,0x0000003a,0x3f000000,0x00020014,0x0000003b,0x0004002b,0x00000023,
0x00000046,0x00000003,0x0004002b,0x00000006,0x00000054,0x00000000,0x0005002c,0x00000007,
0x00000055,0x00000054,0x00000054,0x00040017,0x00000056,0x0000003b,0x00000002,0x0005002c,
0x00000007,0x0000005d,0x0000001c,0x0000001c,0x0006002c,0x0000000f,0x00000063,0x00000054,
0x00000054,0x00000054,0x00090019,0x00000065,0x00000006,0x00000001,0x00000000,0x00000000,
0x00000000,0x00000001,0x00000000,0x0003001b,0x00000066,0x00000065,0x00040020,0x00000067,
0x00000000,0x00000066,0x0004003b,0x00000067,0x00000068,0x00000000,0x00040018,0x00000075,
0x0000000f,0x00000003,0x00040020,0x00000076,0x00000007,0x00000075,0x0004002b,0x00000021,
0x00000078,0x00000001,0x00040020,0x00000079,0x00000009,0x0000001d,0x0004002b,0x00000021,
0x0000007d,0x00000002,0x0004002b,0x00000021,0x00000081,0x00000003,0x00040020,0x00000093,
0x00000007,0x0000000f,0x0004002b,0x00000006,0x0000009b,0xbf800000,0x0004002b,0x00000006,
0x000000ab,0xb8d1b717,0x0004002b,0x00000006,0x000000af,0x3c75c28f,0x0004002b,0x00000006,
0x000000b0,0x3ca3d70a,0x0006002c,0x0000000f,0x000000b1,0x000000af,0x000000af,0x000000b0,
0x00040020,0x000000ed,0x00000001,0x00000007,0x0004003b,0x000000ed,0x000000ee,0x00000001,
0x00040020,0x000000ef,0x00000001,0x00000006,0x0004002b,0x00000006,0x000000f7,0x40000000,
0x0004002b,0x00000006,0x0000015a,0x3cf5c28f,0x00040020,0x00000164,0x00000003,0x0000001d,
0x0004003b,0x00000164,0x00000165,0x00000003,0x00050036,0x00000002,0x00000004,0x00000000,
0x00000003,0x000200f8,0x00000005,0x0004003b,0x00000009,0x000000ec,0x00000007,0x0004003b,
0x00000008,0x000000f4,0x00000007,0x0004003b,0x00000009,0x000000fe,0x00000007,0x0004003b,
0x00000009,0x00000101,0x00000007,0x0004003b,0x00000008,0x0000010c,0x00000007,0x0004003b,
0x00000009,0x00000111,0x00000007,0x0004003b,0x00000008,0x00000114,0x00000007,0x0004003b,
0x00000093,0x00000125,0x00000007,0x0004003b,0x00000008,0x00000129,0x00000007,0x0004003b,
0x00000009,0x0000012b,0x00000007,0x0004003b,0x00000008,0x0000012d,0x00000007,0x0004003b,
0x00000009,0x0000012e,0x00000007,0x0004003b,0x00000008,0x00000133,0x00000007,0x0004003b,
0x00000009,0x00000135,0x00000007,0x0004003b,0x00000008,0x00000137,0x00000007,0x0004003b,
0x00000009,0x00000138,0x00000007,0x0004003b,0x00000008,0x00000140,0x00000007,0x0004003b,
0x00000009,0x00000142,0x00000007,0x0004003b,0x00000008,0x00000144,0x00000007,0x0004003b,
0x00000009,0x00000145,0x00000007,0x0004003b,0x00000008,0x0000014b,0x00000007,0x0004003b,
0x00000009,0x0000014d,0x00000007,0x0004003b,0x00000008,0x0000014f,0x00000007,0x0004003b,
0x00000009,0x00000150,0x00000007,0x0004003b,0x00000008,0x00000153,0x00000007,0x0004003b,
0x00000009,0x00000159,0x00000007,0x00050041,0x000000ef,0x000000f0,0x000000ee,0x00000024,
0x0004003d,0x00000006,0x000000f1,0x000000f0,0x000500b8,0x0000003b,0x000000f2,0x000000f1,
0x0000003a,0x000600a9,0x00000006,0x000000f3,0x000000f2,0x00000054,0x0000001c,0x0003003e,
0x000000ec,0x000000f3,0x00050041,0x000000ef,0x000000f5,0x000000ee,0x00000024,0x0004003d,
0x00000006,0x000000f6,0x000000f5,0x00050085,0x00000006,0x000000f8,0x000000f6,0x000000f7,
0x0004003d,0x00000006,0x000000f9,0x000000ec,0x00050083,0x00000006,0x000000fa,0x000000f8,
0x000000f9,0x00050041,0x000000ef,0x000000fb,0x000000ee,0x0000002b,0x0004003d,0x00000006,
0x000000fc,0x000000fb,0x00050050,0x00000007,0x000000fd,0x000000fa,0x000000fc,0x0003003e,
0x000000f4,0x000000fd,0x0004003d,0x00000006,0x000000ff,0x000000ec,0x000500b8,0x0000003b,
0x00000100,0x000000ff,0x0000003a,0x000300f7,0x00000103,0x00000000,0x000400fa,0x00000100,
0x00000102,0x00000106,0x000200f8,0x00000102,0x00060041,0x00000025,0x00000104,0x00000020,
0x00000036,0x00000024,0x0004003d,0x00000006,0x00000105,0x00000104,0x0003003e,0x00000101,
0x00000105,0x000200f9,0x00000103,0x000200f8,0x00000106,0x00060041,0x00000025,0x00000107,
0x00000020,0x00000036,0x00000024,0x0004003d,0x00000006,0x00000108,0x00000107,0x0004007f,
0x00000006,0x00000109,0x00000108,0x0003003e,0x00000101,0x00000109,0x000200f9,0x00000103,
0x000200f8,0x00000103,0x0004003d,0x00000006,0x0000010a,0x00000101,0x00050081,0x00000006,
0x0000010b,0x0000003a,0x0000010a,0x0003003e,0x000000fe,0x0000010b,0x0004003d,0x00000007,
0x0000010d,0x000000f4,0x0004003d,0x00000006,0x0000010e,0x000000fe,0x00050050,0x00000007,
0x0000010f,0x0000010e,0x0000003a,0x00050083,0x00000007,0x00000110,0x0000010d,0x0000010f,
0x0003003e,0x0000010c,0x00000110,0x00060041,0x00000025,0x00000112,0x00000020,0x00000022,
0x00000037,0x0004003d,0x00000006,0x00000113,0x00000112,0x0003003e,0x00000111,0x00000113,
0x00050041,0x00000009,0x00000115,0x0000010c,0x00000024,0x0004003d,0x00000006,0x00000116,
0x00000115,0x00060041,0x00000025,0x00000117,0x00000020,0x00000036,0x00000046,0x0004003d,
0x00000006,0x00000118,0x00000117,0x00050085,0x00000006,0x00000119,0x00000116,0x00000118,
0x00050041,0x00000009,0x0000011a,0x0000010c,0x0000002b,0x0004003d,0x00000006,0x0000011b,
0x0000011a,0x00050050,0x00000007,0x0000011c,0x00000119,0x0000011b,0x0005008e,0x00000007,
0x0000011d,0x0000011c,0x000000f7,0x0004003d,0x00000006,0x0000011e,0x00000111,0x0005008e,
0x00000007,0x0000011f,0x0000011d,0x0000011e,0x0003003e,0x00000114,0x0000011f,0x00060041,
0x00000025,0x00000120,0x00000020,0x00000022,0x00000046,0x0004003d,0x00000006,0x00000121,
0x00000120,0x000500ba,0x0000003b,0x00000122,0x00000121,0x00000054,0x000300f7,0x00000124,
0x00000000,0x000400fa,0x00000122,0x00000123,0x0000014a,0x000200f8,0x00000123,0x00060041,
0x00000025,0x00000126,0x00000020,0x00000022,0x00000046,0x0004003d,0x00000006,0x00000127,
0x00000126,0x00050083,0x00000006,0x00000128,0x0000001c,0x00000127,0x0004003d,0x00000007,
0x0000012a,0x00000114,0x0003003e,0x00000129,0x0000012a,0x0003003e,0x0000012b,0x00000128,
0x00060039,0x00000007,0x0000012c,0x0000000d,0x00000129,0x0000012b,0x0003003e,0x0000012d,
0x0000012c,0x0004003d,0x00000006,0x0000012f,0x000000ec,0x0003003e,0x0000012e,0x0000012f,
0x00060039,0x0000000f,0x00000130,0x00000013,0x0000012d,0x0000012e,0x00050051,0x00000006,
0x00000131,0x00000130,0x00000000,0x00050041,0x00000009,0x00000132,0x00000125,0x00000024,
0x0003003e,0x00000132,0x00000131,0x0004003d,0x00000007,0x00000134,0x00000114,0x0003003e,
0x00000133,0x00000134,0x0003003e,0x00000135,0x0000001c,0x00060039,0x00000007,0x00000136,
0x0000000d,0x00000133,0x00000135,0x0003003e,0x00000137,0x00000136,0x0004003d,0x00000006,
0x00000139,0x000000ec,0x0003003e,0x00000138,0x00000139,0x00060039,0x0000000f,0x0000013a,
0x00000013,0x00000137,0x00000138,0x00050051,0x00000006,0x0000013b,0x0000013a,0x00000001,
0x00050041,0x00000009,0x0000013c,0x00000125,0x0000002b,0x0003003e,0x0000013c,0x0000013b,
0x00060041,0x00000025,0x0000013d,0x00000020,0x00000022,0x00000046,0x0004003d,0x00000006,
0x0000013e,0x0000013d,0x00050081,0x00000006,0x0000013f,0x0000001c,0x0000013e,0x0004003d,
0x00000007,0x00000141,0x00000114,0x0003003e,0x00000140,0x00000141,0x0003003e,0x00000142,
0x0000013f,0x00060039,0x00000007,0x00000143,0x0000000d,0x00000140,0x00000142,0x0003003e,
0x00000144,0x00000143,0x0004003d,0x00000006,0x00000146,0x000000ec,0x0003003e,0x00000145,
0x00000146,0x00060039,0x0000000f,0x00000147,0x00000013,0x00000144,0x00000145,0x00050051,
0x00000006,0x00000148,0x00000147,0x00000002,0x00050041,0x00000009,0x00000149,0x00000125,
0x00000037,0x0003003e,0x00000149,0x00000148,0x000200f9,0x00000124,0x000200f8,0x0000014a,
0x0004003d,0x00000007,0x0000014c,0x00000114,0x0003003e,0x0000014b,0x0000014c,0x0003003e,
0x0000014d,0x0000001c,0x00060039,0x00000007,0x0000014e,0x0000000d,0x0000014b,0x0000014d,
0x0003003e,0x0000014f,0x0000014e,0x0004003d,0x00000006,0x00000151,0x000000ec,0x0003003e,
0x00000150,0x00000151,0x00060039,0x0000000f,0x00000152,0x00000013,0x0000014f,0x00000150,
0x0003003e,0x00000125,0x00000152,0x000200f9,0x00000124,0x000200f8,0x00000124,0x0004003d,
0x00000007,0x00000154,0x000000f4,0x0004003d,0x00000007,0x00000155,0x000000f4,0x00050050,
0x00000007,0x00000156,0x0000001c,0x0000001c,0x00050083,0x00000007,0x00000157,0x00000156,
0x00000155,0x0007000c,0x00000007,0x00000158,0x00000001,0x00000025,0x00000154,0x00000157,
0x0003003e,0x00000153,0x00000158,0x00050041,0x00000009,0x0000015b,0x00000153,0x00000024,
0x0004003d,0x00000006,0x0000015c,0x0000015b,0x00060041,0x00000025,0x0000015d,0x00000020,
0x00000036,0x00000046,0x0004003d,0x00000006,0x0000015e,0x0000015d,0x00050085,0x00000006,
0x0000015f,0x0000015c,0x0000015e,0x00050041,0x00000009,0x00000160,0x00000153,0x0000002b,
0x0004003d,0x00000006,0x00000161,0x00000160,0x0007000c,0x00000006,0x00000162,0x00000001,
0x00000025,0x0000015f,0x00000161,0x0008000c,0x00000006,0x00000163,0x00000001,0x00000031,
0x00000054,0x0000015a,0x00000162,0x0003003e,0x00000159,0x00000163,0x0004003d,0x0000000f,
0x00000166,0x00000125,0x0004003d,0x00000006,0x00000167,0x00000159,0x0005008e,0x0000000f,
0x00000168,0x00000166,0x00000167,0x00050051,0x00000006,0x00000169,0x00000168,0x00000000,
0x00050051,0x00000006,0x0000016a,0x00000168,0x00000001,0x00050051,0x00000006,0x0000016b,
0x00000168,0x00000002,0x00070050,0x0000001d,0x0000016c,0x00000169,0x0000016a,0x0000016b,
0x0000001c,0x0003003e,0x00000165,0x0000016c,0x000100fd,0x00010038,0x00050036,0x00000007,
0x0000000d,0x00000000,0x0000000a,0x00030037,0x00000008,0x0000000b,0x00030037,0x00000009,
0x0000000c,0x000200f8,0x0000000e,0x0004003b,0x00000009,0x00000015,0x00000007,0x0004003d,
0x00000007,0x00000016,0x0000000b,0x0004003d,0x00000007,0x00000017,0x0000000b,0x00050094,
0x00000006,0x00000018,0x00000016,0x00000017,0x0003003e,0x00000015,0x00000018,0x0004003d,
0x00000007,0x00000019,0x0000000b,0x0004003d,0x00000006,0x0000001a,0x0000000c,0x0005008e,
0x00000007,0x0000001b,0x00000019,0x0000001a,0x00060041,0x00000025,0x00000026,0x00000020,
0x00000022,0x00000024,0x0004003d,0x00000006,0x00000027,0x00000026,0x0004003d,0x00000006,
0x00000028,0x00000015,0x00050085,0x00000006,0x00000029,0x00000027,0x00000028,0x00050081,
0x00000006,0x0000002a,0x0000001c,0x00000029,0x00060041,0x00000025,0x0000002c,0x00000020,
0x00000022,0x0000002b,0x0004003d,0x00000006,0x0000002d,0x0000002c,0x0004003d,0x00000006,
0x0000002e,0x00000015,0x00050085,0x00000006,0x0000002f,0x0000002d,0x0000002e,0x0004003d,
0x00000006,0x00000030,0x00000015,0x00050085,0x00000006,0x00000031,0x0000002f,0x00000030,
0x00050081,0x00000006,0x00000032,0x0000002a,0x00000031,0x0005008e,0x00000007,0x00000033,
0x0000001b,0x00000032,0x000200fe,0x00000033,0x00010038,0x00050036,0x0000000f,0x00000013,
0x00000000,0x00000010,0x00030037,0x00000008,0x00000011,0x00030037,0x00000009,0x00000012,
0x000200f8,0x00000014,0x0004003b,0x00000009,0x0000003f,0x00000007,0x0004003b,0x00000008,
0x00000042,0x00000007,0x0004003b,0x00000076,0x00000077,0x00000007,0x0004003b,0x00000093,
0x00000094,0x00000007,0x0004003b,0x00000093,0x0000009f,0x00000007,0x0004003b,0x00000009,
0x000000b3,0x00000007,0x0004003b,0x00000093,0x000000bd,0x00000007,0x0004003b,0x00000009,
0x000000c3,0x00000007,0x0004003b,0x00000009,0x000000c6,0x00000007,0x0004003b,0x00000008,
0x000000cb,0x00000007,0x00060041,0x00000025,0x00000038,0x00000020,0x00000036,0x00000037,
0x0004003d,0x00000006,0x00000039,0x00000038,0x000500ba,0x0000003b,0x0000003c,0x00000039,
0x0000003a,0x000300f7,0x0000003e,0x00000000,0x000400fa,0x0000003c,0x0000003d,0x0000003e,
0x000200f8,0x0000003d,0x00060041,0x00000025,0x00000040,0x00000020,0x00000022,0x00000037,
0x0004003d,0x00000006,0x00000041,0x00000040,0x0003003e,0x0000003f,0x00000041,0x00050041,
0x00000009,0x00000043,0x00000011,0x00000024,0x0004003d,0x00000006,0x00000044,0x00000043,
0x0004003d,0x00000006,0x00000045,0x0000003f,0x00060041,0x00000025,0x00000047,0x00000020,
0x00000036,0x00000046,0x0004003d,0x00000006,0x00000048,0x00000047,0x00050085,0x00000006,
0x00000049,0x00000045,0x00000048,0x00050088,0x00000006,0x0000004a,0x00000044,0x00000049,
0x00050041,0x00000009,0x0000004b,0x00000011,0x0000002b,0x0004003d,0x00000006,0x0000004c,
0x0000004b,0x0004003d,0x00000006,0x0000004d,0x0000003f,0x00050088,0x00000006,0x0000004e,
0x0000004c,0x0000004d,0x00050050,0x00000007,0x0000004f,0x0000004a,0x0000004e,0x0005008e,
0x00000007,0x00000050,0x0000004f,0x0000003a,0x00050050,0x00000007,0x00000051,0x0000003a,
0x0000003a,0x00050081,0x00000007,0x00000052,0x00000050,0x00000051,0x0003003e,0x00000042,
0x00000052,0x0004003d,0x00000007,0x00000053,0x00000042,0x000500b8,0x00000056,0x00000057,
0x00000053,0x00000055,0x0004009a,0x0000003b,0x00000058,0x00000057,0x000400a8,0x0000003b,
0x00000059,0x00000058,0x000300f7,0x0000005b,0x00000000,0x000400fa,0x00000059,0x0000005a,
0x0000005b,0x000200f8,0x0000005a,0x0004003d,0x00000007,0x0000005c,0x00000042,0x000500ba,
0x00000056,0x0000005e,0x0000005c,0x0000005d,0x0004009a,0x0000003b,0x0000005f,0x0000005e,
0x000200f9,0x0000005b,0x000200f8,0x0000005b,0x000700f5,0x0000003b,0x00000060,0x00000058,
0x0000003d,0x0000005f,0x0000005a,0x000300f7,0x00000062,0x00000000,0x000400fa,0x00000060,
0x00000061,0x00000062,0x000200f8,0x00000061,0x000200fe,0x00000063,0x000200f8,0x00000062,
0x0004003d,0x00000066,0x00000069,0x00000068,0x00050041,0x00000009,0x0000006a,0x00000042,
0x00000024,0x0004003d,0x00000006,0x0000006b,0x0000006a,0x0004003d,0x00000006,0x0000006c,
0x00000012,0x00050081,0x00000006,0x0000006d,0x0000006b,0x0000006c,0x00050085,0x00000006,
0x0000006e,0x0000006d,0x0000003a,0x00050041,0x00000009,0x0000006f,0x00000042,0x0000002b,
0x0004003d,0x00000006,0x00000070,0x0000006f,0x00050050,0x00000007,0x00000071,0x0000006e,
0x00000070,0x00050057,0x0000001d,0x00000072,0x00000069,0x00000071,0x0008004f,0x0000000f,
0x00000073,0x00000072,0x00000072,0x00000000,0x00000001,0x00000002,0x000200fe,0x00000073,
0x000200f8,0x0000003e,0x00050041,0x00000079,0x0000007a,0x00000020,0x00000078,0x0004003d,
0x0000001d,0x0000007b,0x0000007a,0x0008004f,0x0000000f,0x0000007c,0x0000007b,0x0000007b,
0x00000000,0x00000001,0x00000002,0x00050041,0x00000079,0x0000007e,0x00000020,0x0000007d,
0x0004003d,0x0000001d,0x0000007f,0x0000007e,0x0008004f,0x0000000f,0x00000080,0x0000007f,
0x0000007f,0x00000000,0x00000001,0x00000002,0x00050041,0x00000079,0x00000082,0x00000020,
0x00000081,0x0004003d,0x0000001d,0x00000083,0x00000082,0x0008004f,0x0000000f,0x00000084,
0x00000083,0x00000083,0x00000000,0x00000001,0x00000002,0x00050051,0x00000006,0x00000085,
0x0000007c,0x00000000,0x00050051,0x00000006,0x00000086,0x0000007c,0x00000001,0x00050051,
0x00000006,0x00000087,0x0000007c,0x00000002,0x00050051,0x00000006,0x00000088,0x00000080,
0x00000000,0x00050051,0x00000006,0x00000089,0x00000080,0x00000001,0x00050051,0x00000006,
0x0000008a,0x00000080,0x00000002,0x00050051,0x00000006,0x0000008b,0x00000084,0x00000000,
0x00050051,0x00000006,0x0000008c,0x00000084,0x00000001,0x00050051,0x00000006,0x0000008d,
0x00000084,0x00000002,0x00060050,0x0000000f,0x0000008e,0x00000085,0x00000086,0x00000087,
0x00060050,0x0000000f,0x0000008f,0x00000088,0x00000089,0x0000008a,0x00060050,0x0000000f,
0x00000090,0x0000008b,0x0000008c,0x0000008d,0x00060050,0x00000075,0x00000091,0x0000008e,
0x0000008f,0x00000090,0x00040054,0x00000075,0x00000092,0x00000091,0x0003003e,0x00000077,
0x00000092,0x0004003d,0x00000075,0x00000095,0x00000077,0x00050041,0x00000009,0x00000096,
0x00000011,0x00000024,0x0004003d,0x00000006,0x00000097,0x00000096,0x00050041,0x00000009,
0x00000098,0x00000011,0x0000002b,0x0004003d,0x00000006,0x00000099,0x00000098,0x0004007f,
0x00000006,0x0000009a,0x00000099,0x00060050,0x0000000f,0x0000009c,0x00000097,0x0000009a,
0x0000009b,0x0006000c,0x0000000f,0x0000009d,0x00000001,0x00000045,0x0000009c,0x00050091,
0x0000000f,0x0000009e,0x00000095,0x0000009d,0x0003003e,0x00000094,0x0000009e,0x0004003d,
0x00000075,0x000000a0,0x00000077,0x0004003d,0x00000006,0x000000a1,0x00000012,0x000500b8,
0x0000003b,0x000000a2,0x000000a1,0x0000003a,0x000600a9,0x00000006,0x000000a3,0x000000a2,
0x0000009b,0x0000001c,0x00060041,0x00000025,0x000000a4,0x00000020,0x00000078,0x00000046,
0x0004003d,0x00000006,0x000000a5,0x000000a4,0x00050085,0x00000006,0x000000a6,0x000000a3,
0x000000a5,0x00060050,0x0000000f,0x000000a7,0x000000a6,0x00000054,0x00000054,0x00050091,
0x0000000f,0x000000a8,0x000000a0,0x000000a7,0x0003003e,0x0000009f,0x000000a8,0x00050041,
0x00000009,0x000000a9,0x00000094,0x00000037,0x0004003d,0x00000006,0x000000aa,0x000000a9,
0x000500ba,0x0000003b,0x000000ac,0x000000aa,0x000000ab,0x000300f7,0x000000ae,0x00000000,
0x000400fa,0x000000ac,0x000000ad,0x000000ae,0x000200f8,0x000000ad,0x000200fe,0x000000b1,
0x000200f8,0x000000ae,0x00060041,0x00000025,0x000000b4,0x00000020,0x0000007d,0x00000046,
0x0004003d,0x00000006,0x000000b5,0x000000b4,0x0004007f,0x00000006,0x000000b6,0x000000b5,
0x00050041,0x00000009,0x000000b7,0x0000009f,0x00000037,0x0004003d,0x00000006,0x000000b8,
0x000000b7,0x00050083,0x00000006,0x000000b9,0x000000b6,0x000000b8,0x00050041,0x00000009,
0x000000ba,0x00000094,0x00000037,0x0004003d,0x00000006,0x000000bb,0x000000ba,0x00050088,
0x00000006,0x000000bc,0x000000b9,0x000000bb,0x0003003e,0x000000b3,0x000000bc,0x0004003d,
0x0000000f,0x000000be,0x0000009f,0x0004003d,0x0000000f,0x000000bf,0x00000094,0x0004003d,
0x00000006,0x000000c0,0x000000b3,0x0005008e,0x0000000f,0x000000c1,0x000000bf,0x000000c0,
0x00050081,0x0000000f,0x000000c2,0x000000be,0x000000c1,0x0003003e,0x000000bd,0x000000c2,
0x00060041,0x00000025,0x000000c4,0x00000020,0x00000081,0x00000046,0x0004003d,0x00000006,
0x000000c5,0x000000c4,0x0003003e,0x000000c3,0x000000c5,0x0004003d,0x00000006,0x000000c7,
0x000000c3,0x00060041,0x00000025,0x000000c8,0x00000020,0x00000036,0x0000002b,0x0004003d,
0x00000006,0x000000c9,0x000000c8,0x00050088,0x00000006,0x000000ca,0x000000c7,0x000000c9,
0x0003003e,0x000000c6,0x000000ca,0x00050041,0x00000009,0x000000cc,0x000000bd,0x00000024,
0x0004003d,0x00000006,0x000000cd,0x000000cc,0x0004003d,0x00000006,0x000000ce,0x000000c3,
0x00050088,0x00000006,0x000000cf,0x000000cd,0x000000ce,0x00050041,0x00000009,0x000000d0,
0x000000bd,0x0000002b,0x0004003d,0x00000006,0x000000d1,0x000000d0,0x0004007f,0x00000006,
0x000000d2,0x000000d1,0x0004003d,0x00000006,0x000000d3,0x000000c6,0x00050088,0x00000006,
0x000000d4,0x000000d2,0x000000d3,0x00050050,0x00000007,0x000000d5,0x000000cf,0x000000d4,
0x0005008e,0x00000007,0x000000d6,0x000000d5,0x0000003a,0x00050050,0x00000007,0x000000d7,
0x0000003a,0x0000003a,0x00050081,0x00000007,0x000000d8,0x000000d6,0x000000d7,0x0003003e,
0x000000cb,0x000000d8,0x0004003d,0x00000007,0x000000d9,0x000000cb,0x000500b8,0x00000056,
0x000000da,0x000000d9,0x00000055,0x0004009a,0x0000003b,0x000000db,0x000000da,0x000400a8,
0x0000003b,0x000000dc,0x000000db,0x000300f7,0x000000de,0x00000000,0x000400fa,0x000000dc,
0x000000dd,0x000000de,0x000200f8,0x000000dd,0x0004003d,0x00000007,0x000000df,0x000000cb,
0x000500ba,0x00000056,0x000000e0,0x000000df,0x0000005d,0x0004009a,0x0000003b,0x000000e1,
0x000000e0,0x000200f9,0x000000de,0x000200f8,0x000000de,0x000700f5,0x0000003b,0x000000e2,
0x000000db,0x000000ae,0x000000e1,0x000000dd,0x000300f7,0x000000e4,0x00000000,0x000400fa,
0x000000e2,0x000000e3,0x000000e4,0x000200f8,0x000000e3,0x000200fe,0x000000b1,0x000200f8,
0x000000e4,0x0004003d,0x00000066,0x000000e6,0x00000068,0x0004003d,0x00000007,0x000000e7,
0x000000cb,0x00050057,0x0000001d,0x000000e8,0x000000e6,0x000000e7,0x0008004f,0x0000000f,
0x000000e9,0x000000e8,0x000000e8,0x00000000,0x00000001,0x00000002,0x000200fe,0x000000e9,
0x00010038
0x736e656c,0x00000000,0x00050006,0x0000001e,0x00000005,0x77656976,0x00000000,0x00050006,
0x0000001e,0x00000006,0x72747865,0x00000061,0x00030005,0x00000020,0x00006370,0x00040005,
0x0000003f,0x486e6174,0x00000000,0x00030005,0x00000042,0x00000073,0x00040005,0x00000068,
0x72756f53,0x00006563,0x00030005,0x00000077,0x00000052,0x00030005,0x00000094,0x00000064,
0x00030005,0x0000009f,0x0000006f,0x00030005,0x000000b3,0x00000074,0x00030005,0x000000bd,
0x00746968,0x00040005,0x000000c3,0x666c6168,0x00000057,0x00040005,0x000000c6,0x666c6168,
0x00000048,0x00030005,0x000000cb,0x00000073,0x00040005,0x000000ed,0x6f6e6f6d,0x00000000,
0x00030005,0x000000f2,0x00657965,0x00030005,0x000000f8,0x00007675,0x00040005,0x000000ff,
0x61636f6c,0x0000006c,0x00030005,0x00000110,0x00007863,0x00030005,0x00000124,0x00000070,
0x00040005,0x00000129,0x486e6174,0x00000000,0x00030005,0x0000012c,0x00000074,0x00030005,
0x0000013d,0x00000063,0x00040005,0x00000141,0x61726170,0x0000006d,0x00040005,0x00000143,
0x61726170,0x0000006d,0x00040005,0x00000145,0x61726170,0x0000006d,0x00040005,0x00000146,
0x61726170,0x0000006d,0x00040005,0x0000014b,0x61726170,0x0000006d,0x00040005,0x0000014d,
0x61726170,0x0000006d,0x00040005,0x0000014f,0x61726170,0x0000006d,0x00040005,0x00000150,
0x61726170,0x0000006d,0x00040005,0x00000158,0x61726170,0x0000006d,0x00040005,0x0000015a,
0x61726170,0x0000006d,0x00040005,0x0000015c,0x61726170,0x0000006d,0x00040005,0x0000015d,
0x61726170,0x0000006d,0x00040005,0x00000163,0x61726170,0x0000006d,0x00040005,0x00000165,
0x61726170,0x0000006d,0x00040005,0x00000167,0x61726170,0x0000006d,0x00040005,0x00000168,
0x61726170,0x0000006d,0x00030005,0x0000016b,0x00000065,0x00040005,0x00000171,0x65646166,
0x00000000,0x00050005,0x0000017d,0x4374756f,0x726f6c6f,0x00000000,0x00030047,0x0000001e,
0x00000002,0x00050048,0x0000001e,0x00000000,0x00000023,0x00000000,0x00050048,0x0000001e,
0x00000001,0x00000023,0x00000010,0x00050048,0x0000001e,0x00000002,0x00000023,0x00000020,
0x00050048,0x0000001e,0x00000003,0x00000023,0x00000030,0x00050048,0x0000001e,0x00000004,
0x00000023,0x00000040,0x00050048,0x0000001e,0x00000005,0x00000023,0x00000050,0x00050048,
0x0000001e,0x00000006,0x00000023,0x00000060,0x00040047,0x00000068,0x00000021,0x00000000,
0x00040047,0x00000068,0x00000022,0x00000000,0x00040047,0x000000f8,0x0000001e,0x00000000,
0x00040047,0x0000017d,0x0000001e,0x00000000,0x00020013,0x00000002,0x00030021,0x00000003,
0x00000002,0x00030016,0x00000006,0x00000020,0x00040017,0x00000007,0x00000006,0x00000002,
0x00040020,0x00000008,0x00000007,0x00000007,0x00040020,0x00000009,0x00000007,0x00000006,
0x00050021,0x0000000a,0x00000007,0x00000008,0x00000009,0x00040017,0x0000000f,0x00000006,
0x00000003,0x00050021,0x00000010,0x0000000f,0x00000008,0x00000009,0x0004002b,0x00000006,
0x0000001c,0x3f800000,0x00040017,0x0000001d,0x00000006,0x00000004,0x0009001e,0x0000001e,
0x0000001d,0x0000001d,0x0000001d,0x0000001d,0x0000001d,0x0000001d,0x0000001d,0x00040020,
0x0000001f,0x00000009,0x0000001e,0x0004003b,0x0000001f,0x00000020,0x00000009,0x00040015,
0x00000021,0x00000020,0x00000001,0x0004002b,0x00000021,0x00000022,0x00000004,0x00040015,
0x00000023,0x00000020,0x00000000,0x0004002b,0x00000023,0x00000024,0x00000000,0x00040020,
0x00000025,0x00000009,0x00000006,0x0004002b,0x00000023,0x0000002b,0x00000001,0x0004002b,
0x00000021,0x00000036,0x00000005,0x0004002b,0x00000023,0x00000037,0x00000002,0x0004002b,
0x00000006,0x0000003a,0x3f000000,0x00020014,0x0000003b,0x0004002b,0x00000023,0x00000046,
0x00000003,0x0004002b,0x00000006,0x00000054,0x00000000,0x0005002c,0x00000007,0x00000055,
0x00000054,0x00000054,0x00040017,0x00000056,0x0000003b,0x00000002,0x0005002c,0x00000007,
0x0000005d,0x0000001c,0x0000001c,0x0006002c,0x0000000f,0x00000063,0x00000054,0x00000054,
0x00000054,0x00090019,0x00000065,0x00000006,0x00000001,0x00000000,0x00000000,0x00000000,
0x00000001,0x00000000,0x0003001b,0x00000066,0x00000065,0x00040020,0x00000067,0x00000000,
0x00000066,0x0004003b,0x00000067,0x00000068,0x00000000,0x00040018,0x00000075,0x0000000f,
0x00000003,0x00040020,0x00000076,0x00000007,0x00000075,0x0004002b,0x00000021,0x00000078,
0x00000001,0x00040020,0x00000079,0x00000009,0x0000001d,0x0004002b,0x00000021,0x0000007d,
0x00000002,0x0004002b,0x00000021,0x00000081,0x00000003,0x00040020,0x00000093,0x00000007,
0x0000000f,0x0004002b,0x00000006,0x0000009b,0xbf800000,0x0004002b,0x00000006,0x000000ab,
0xb8d1b717,0x0004002b,0x00000006,0x000000af,0x3c75c28f,0x0004002b,0x00000006,0x000000b0,
0x3ca3d70a,0x0006002c,0x0000000f,0x000000b1,0x000000af,0x000000af,0x000000b0,0x00040020,
0x000000ec,0x00000007,0x0000003b,0x0004002b,0x00000021,0x000000ee,0x00000006,0x00040020,
0x000000f7,0x00000001,0x00000007,0x0004003b,0x000000f7,0x000000f8,0x00000001,0x00040020,
0x000000f9,0x00000001,0x00000006,0x0004002b,0x00000006,0x00000108,0x40000000,0x0004002b,
0x00000006,0x00000172,0x3cf5c28f,0x00040020,0x0000017c,0x00000003,0x0000001d,0x0004003b,
0x0000017c,0x0000017d,0x00000003,0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,
0x000200f8,0x00000005,0x0004003b,0x000000ec,0x000000ed,0x00000007,0x0004003b,0x00000009,
0x000000f2,0x00000007,0x0004003b,0x00000008,0x000000ff,0x00000007,0x0004003b,0x00000008,
0x00000101,0x00000007,0x0004003b,0x00000009,0x00000110,0x00000007,0x0004003b,0x00000009,
0x00000112,0x00000007,0x0004003b,0x00000009,0x00000118,0x00000007,0x0004003b,0x00000008,
0x00000124,0x00000007,0x0004003b,0x00000009,0x00000129,0x00000007,0x0004003b,0x00000008,
0x0000012c,0x00000007,0x0004003b,0x00000093,0x0000013d,0x00000007,0x0004003b,0x00000008,
0x00000141,0x00000007,0x0004003b,0x00000009,0x00000143,0x00000007,0x0004003b,0x00000008,
0x00000145,0x00000007,0x0004003b,0x00000009,0x00000146,0x00000007,0x0004003b,0x00000008,
0x0000014b,0x00000007,0x0004003b,0x00000009,0x0000014d,0x00000007,0x0004003b,0x00000008,
0x0000014f,0x00000007,0x0004003b,0x00000009,0x00000150,0x00000007,0x0004003b,0x00000008,
0x00000158,0x00000007,0x0004003b,0x00000009,0x0000015a,0x00000007,0x0004003b,0x00000008,
0x0000015c,0x00000007,0x0004003b,0x00000009,0x0000015d,0x00000007,0x0004003b,0x00000008,
0x00000163,0x00000007,0x0004003b,0x00000009,0x00000165,0x00000007,0x0004003b,0x00000008,
0x00000167,0x00000007,0x0004003b,0x00000009,0x00000168,0x00000007,0x0004003b,0x00000008,
0x0000016b,0x00000007,0x0004003b,0x00000009,0x00000171,0x00000007,0x00060041,0x00000025,
0x000000ef,0x00000020,0x000000ee,0x00000024,0x0004003d,0x00000006,0x000000f0,0x000000ef,
0x000500ba,0x0000003b,0x000000f1,0x000000f0,0x0000003a,0x0003003e,0x000000ed,0x000000f1,
0x0004003d,0x0000003b,0x000000f3,0x000000ed,0x000400a8,0x0000003b,0x000000f4,0x000000f3,
0x000300f7,0x000000f6,0x00000000,0x000400fa,0x000000f4,0x000000f5,0x000000f6,0x000200f8,
0x000000f5,0x00050041,0x000000f9,0x000000fa,0x000000f8,0x00000024,0x0004003d,0x00000006,
0x000000fb,0x000000fa,0x000500b8,0x0000003b,0x000000fc,0x000000fb,0x0000003a,0x000200f9,
0x000000f6,0x000200f8,0x000000f6,0x000700f5,0x0000003b,0x000000fd,0x000000f3,0x00000005,
0x000000fc,0x000000f5,0x000600a9,0x00000006,0x000000fe,0x000000fd,0x00000054,0x0000001c,
0x0003003e,0x000000f2,0x000000fe,0x0004003d,0x0000003b,0x00000100,0x000000ed,0x000300f7,
0x00000103,0x00000000,0x000400fa,0x00000100,0x00000102,0x00000105,0x000200f8,0x00000102,
0x0004003d,0x00000007,0x00000104,0x000000f8,0x0003003e,0x00000101,0x00000104,0x000200f9,
0x00000103,0x000200f8,0x00000105,0x00050041,0x000000f9,0x00000106,0x000000f8,0x00000024,
0x0004003d,0x00000006,0x00000107,0x00000106,0x00050085,0x00000006,0x00000109,0x00000107,
0x00000108,0x0004003d,0x00000006,0x0000010a,0x000000f2,0x00050083,0x00000006,0x0000010b,
0x00000109,0x0000010a,0x00050041,0x000000f9,0x0000010c,0x000000f8,0x0000002b,0x0004003d,
0x00000006,0x0000010d,0x0000010c,0x00050050,0x00000007,0x0000010e,0x0000010b,0x0000010d,
0x0003003e,0x00000101,0x0000010e,0x000200f9,0x00000103,0x000200f8,0x00000103,0x0004003d,
0x00000007,0x0000010f,0x00000101,0x0003003e,0x000000ff,0x0000010f,0x0004003d,0x0000003b,
0x00000111,0x000000ed,0x000300f7,0x00000114,0x00000000,0x000400fa,0x00000111,0x00000113,
0x00000115,0x000200f8,0x00000113,0x0003003e,0x00000112,0x0000003a,0x000200f9,0x00000114,
0x000200f8,0x00000115,0x0004003d,0x00000006,0x00000116,0x000000f2,0x000500b8,0x0000003b,
0x00000117,0x00000116,0x0000003a,0x000300f7,0x0000011a,0x00000000,0x000400fa,0x00000117,
0x00000119,0x0000011d,0x000200f8,0x00000119,0x00060041,0x00000025,0x0000011b,0x00000020,
0x00000036,0x00000024,0x0004003d,0x00000006,0x0000011c,0x0000011b,0x0003003e,0x00000118,
0x0000011c,0x000200f9,0x0000011a,0x000200f8,0x0000011d,0x00060041,0x00000025,0x0000011e,
0x00000020,0x00000036,0x00000024,0x0004003d,0x00000006,0x0000011f,0x0000011e,0x0004007f,
0x00000006,0x00000120,0x0000011f,0x0003003e,0x00000118,0x00000120,0x000200f9,0x0000011a,
0x000200f8,0x0000011a,0x0004003d,0x00000006,0x00000121,0x00000118,0x00050081,0x00000006,
0x00000122,0x0000003a,0x00000121,0x0003003e,0x00000112,0x00000122,0x000200f9,0x00000114,
0x000200f8,0x00000114,0x0004003d,0x00000006,0x00000123,0x00000112,0x0003003e,0x00000110,
0x00000123,0x0004003d,0x00000007,0x00000125,0x000000ff,0x0004003d,0x00000006,0x00000126,
0x00000110,0x00050050,0x00000007,0x00000127,0x00000126,0x0000003a,0x00050083,0x00000007,
0x00000128,0x00000125,0x00000127,0x0003003e,0x00000124,0x00000128,0x00060041,0x00000025,
0x0000012a,0x00000020,0x00000022,0x00000037,0x0004003d,0x00000006,0x0000012b,0x0000012a,
0x0003003e,0x00000129,0x0000012b,0x00050041,0x00000009,0x0000012d,0x00000124,0x00000024,
0x0004003d,0x00000006,0x0000012e,0x0000012d,0x00060041,0x00000025,0x0000012f,0x00000020,
0x00000036,0x00000046,0x0004003d,0x00000006,0x00000130,0x0000012f,0x00050085,0x00000006,
0x00000131,0x0000012e,0x00000130,0x00050041,0x00000009,0x00000132,0x00000124,0x0000002b,
0x0004003d,0x00000006,0x00000133,0x00000132,0x00050050,0x00000007,0x00000134,0x00000131,
0x00000133,0x0005008e,0x00000007,0x00000135,0x00000134,0x00000108,0x0004003d,0x00000006,
0x00000136,0x00000129,0x0005008e,0x00000007,0x00000137,0x00000135,0x00000136,0x0003003e,
0x0000012c,0x00000137,0x00060041,0x00000025,0x00000138,0x00000020,0x00000022,0x00000046,
0x0004003d,0x00000006,0x00000139,0x00000138,0x000500ba,0x0000003b,0x0000013a,0x00000139,
0x00000054,0x000300f7,0x0000013c,0x00000000,0x000400fa,0x0000013a,0x0000013b,0x00000162,
0x000200f8,0x0000013b,0x00060041,0x00000025,0x0000013e,0x00000020,0x00000022,0x00000046,
0x0004003d,0x00000006,0x0000013f,0x0000013e,0x00050083,0x00000006,0x00000140,0x0000001c,
0x0000013f,0x0004003d,0x00000007,0x00000142,0x0000012c,0x0003003e,0x00000141,0x00000142,
0x0003003e,0x00000143,0x00000140,0x00060039,0x00000007,0x00000144,0x0000000d,0x00000141,
0x00000143,0x0003003e,0x00000145,0x00000144,0x0004003d,0x00000006,0x00000147,0x000000f2,
0x0003003e,0x00000146,0x00000147,0x00060039,0x0000000f,0x00000148,0x00000013,0x00000145,
0x00000146,0x00050051,0x00000006,0x00000149,0x00000148,0x00000000,0x00050041,0x00000009,
0x0000014a,0x0000013d,0x00000024,0x0003003e,0x0000014a,0x00000149,0x0004003d,0x00000007,
0x0000014c,0x0000012c,0x0003003e,0x0000014b,0x0000014c,0x0003003e,0x0000014d,0x0000001c,
0x00060039,0x00000007,0x0000014e,0x0000000d,0x0000014b,0x0000014d,0x0003003e,0x0000014f,
0x0000014e,0x0004003d,0x00000006,0x00000151,0x000000f2,0x0003003e,0x00000150,0x00000151,
0x00060039,0x0000000f,0x00000152,0x00000013,0x0000014f,0x00000150,0x00050051,0x00000006,
0x00000153,0x00000152,0x00000001,0x00050041,0x00000009,0x00000154,0x0000013d,0x0000002b,
0x0003003e,0x00000154,0x00000153,0x00060041,0x00000025,0x00000155,0x00000020,0x00000022,
0x00000046,0x0004003d,0x00000006,0x00000156,0x00000155,0x00050081,0x00000006,0x00000157,
0x0000001c,0x00000156,0x0004003d,0x00000007,0x00000159,0x0000012c,0x0003003e,0x00000158,
0x00000159,0x0003003e,0x0000015a,0x00000157,0x00060039,0x00000007,0x0000015b,0x0000000d,
0x00000158,0x0000015a,0x0003003e,0x0000015c,0x0000015b,0x0004003d,0x00000006,0x0000015e,
0x000000f2,0x0003003e,0x0000015d,0x0000015e,0x00060039,0x0000000f,0x0000015f,0x00000013,
0x0000015c,0x0000015d,0x00050051,0x00000006,0x00000160,0x0000015f,0x00000002,0x00050041,
0x00000009,0x00000161,0x0000013d,0x00000037,0x0003003e,0x00000161,0x00000160,0x000200f9,
0x0000013c,0x000200f8,0x00000162,0x0004003d,0x00000007,0x00000164,0x0000012c,0x0003003e,
0x00000163,0x00000164,0x0003003e,0x00000165,0x0000001c,0x00060039,0x00000007,0x00000166,
0x0000000d,0x00000163,0x00000165,0x0003003e,0x00000167,0x00000166,0x0004003d,0x00000006,
0x00000169,0x000000f2,0x0003003e,0x00000168,0x00000169,0x00060039,0x0000000f,0x0000016a,
0x00000013,0x00000167,0x00000168,0x0003003e,0x0000013d,0x0000016a,0x000200f9,0x0000013c,
0x000200f8,0x0000013c,0x0004003d,0x00000007,0x0000016c,0x000000ff,0x0004003d,0x00000007,
0x0000016d,0x000000ff,0x00050050,0x00000007,0x0000016e,0x0000001c,0x0000001c,0x00050083,
0x00000007,0x0000016f,0x0000016e,0x0000016d,0x0007000c,0x00000007,0x00000170,0x00000001,
0x00000025,0x0000016c,0x0000016f,0x0003003e,0x0000016b,0x00000170,0x00050041,0x00000009,
0x00000173,0x0000016b,0x00000024,0x0004003d,0x00000006,0x00000174,0x00000173,0x00060041,
0x00000025,0x00000175,0x00000020,0x00000036,0x00000046,0x0004003d,0x00000006,0x00000176,
0x00000175,0x00050085,0x00000006,0x00000177,0x00000174,0x00000176,0x00050041,0x00000009,
0x00000178,0x0000016b,0x0000002b,0x0004003d,0x00000006,0x00000179,0x00000178,0x0007000c,
0x00000006,0x0000017a,0x00000001,0x00000025,0x00000177,0x00000179,0x0008000c,0x00000006,
0x0000017b,0x00000001,0x00000031,0x00000054,0x00000172,0x0000017a,0x0003003e,0x00000171,
0x0000017b,0x0004003d,0x0000000f,0x0000017e,0x0000013d,0x0004003d,0x00000006,0x0000017f,
0x00000171,0x0005008e,0x0000000f,0x00000180,0x0000017e,0x0000017f,0x00050051,0x00000006,
0x00000181,0x00000180,0x00000000,0x00050051,0x00000006,0x00000182,0x00000180,0x00000001,
0x00050051,0x00000006,0x00000183,0x00000180,0x00000002,0x00070050,0x0000001d,0x00000184,
0x00000181,0x00000182,0x00000183,0x0000001c,0x0003003e,0x0000017d,0x00000184,0x000100fd,
0x00010038,0x00050036,0x00000007,0x0000000d,0x00000000,0x0000000a,0x00030037,0x00000008,
0x0000000b,0x00030037,0x00000009,0x0000000c,0x000200f8,0x0000000e,0x0004003b,0x00000009,
0x00000015,0x00000007,0x0004003d,0x00000007,0x00000016,0x0000000b,0x0004003d,0x00000007,
0x00000017,0x0000000b,0x00050094,0x00000006,0x00000018,0x00000016,0x00000017,0x0003003e,
0x00000015,0x00000018,0x0004003d,0x00000007,0x00000019,0x0000000b,0x0004003d,0x00000006,
0x0000001a,0x0000000c,0x0005008e,0x00000007,0x0000001b,0x00000019,0x0000001a,0x00060041,
0x00000025,0x00000026,0x00000020,0x00000022,0x00000024,0x0004003d,0x00000006,0x00000027,
0x00000026,0x0004003d,0x00000006,0x00000028,0x00000015,0x00050085,0x00000006,0x00000029,
0x00000027,0x00000028,0x00050081,0x00000006,0x0000002a,0x0000001c,0x00000029,0x00060041,
0x00000025,0x0000002c,0x00000020,0x00000022,0x0000002b,0x0004003d,0x00000006,0x0000002d,
0x0000002c,0x0004003d,0x00000006,0x0000002e,0x00000015,0x00050085,0x00000006,0x0000002f,
0x0000002d,0x0000002e,0x0004003d,0x00000006,0x00000030,0x00000015,0x00050085,0x00000006,
0x00000031,0x0000002f,0x00000030,0x00050081,0x00000006,0x00000032,0x0000002a,0x00000031,
0x0005008e,0x00000007,0x00000033,0x0000001b,0x00000032,0x000200fe,0x00000033,0x00010038,
0x00050036,0x0000000f,0x00000013,0x00000000,0x00000010,0x00030037,0x00000008,0x00000011,
0x00030037,0x00000009,0x00000012,0x000200f8,0x00000014,0x0004003b,0x00000009,0x0000003f,
0x00000007,0x0004003b,0x00000008,0x00000042,0x00000007,0x0004003b,0x00000076,0x00000077,
0x00000007,0x0004003b,0x00000093,0x00000094,0x00000007,0x0004003b,0x00000093,0x0000009f,
0x00000007,0x0004003b,0x00000009,0x000000b3,0x00000007,0x0004003b,0x00000093,0x000000bd,
0x00000007,0x0004003b,0x00000009,0x000000c3,0x00000007,0x0004003b,0x00000009,0x000000c6,
0x00000007,0x0004003b,0x00000008,0x000000cb,0x00000007,0x00060041,0x00000025,0x00000038,
0x00000020,0x00000036,0x00000037,0x0004003d,0x00000006,0x00000039,0x00000038,0x000500ba,
0x0000003b,0x0000003c,0x00000039,0x0000003a,0x000300f7,0x0000003e,0x00000000,0x000400fa,
0x0000003c,0x0000003d,0x0000003e,0x000200f8,0x0000003d,0x00060041,0x00000025,0x00000040,
0x00000020,0x00000022,0x00000037,0x0004003d,0x00000006,0x00000041,0x00000040,0x0003003e,
0x0000003f,0x00000041,0x00050041,0x00000009,0x00000043,0x00000011,0x00000024,0x0004003d,
0x00000006,0x00000044,0x00000043,0x0004003d,0x00000006,0x00000045,0x0000003f,0x00060041,
0x00000025,0x00000047,0x00000020,0x00000036,0x00000046,0x0004003d,0x00000006,0x00000048,
0x00000047,0x00050085,0x00000006,0x00000049,0x00000045,0x00000048,0x00050088,0x00000006,
0x0000004a,0x00000044,0x00000049,0x00050041,0x00000009,0x0000004b,0x00000011,0x0000002b,
0x0004003d,0x00000006,0x0000004c,0x0000004b,0x0004003d,0x00000006,0x0000004d,0x0000003f,
0x00050088,0x00000006,0x0000004e,0x0000004c,0x0000004d,0x00050050,0x00000007,0x0000004f,
0x0000004a,0x0000004e,0x0005008e,0x00000007,0x00000050,0x0000004f,0x0000003a,0x00050050,
0x00000007,0x00000051,0x0000003a,0x0000003a,0x00050081,0x00000007,0x00000052,0x00000050,
0x00000051,0x0003003e,0x00000042,0x00000052,0x0004003d,0x00000007,0x00000053,0x00000042,
0x000500b8,0x00000056,0x00000057,0x00000053,0x00000055,0x0004009a,0x0000003b,0x00000058,
0x00000057,0x000400a8,0x0000003b,0x00000059,0x00000058,0x000300f7,0x0000005b,0x00000000,
0x000400fa,0x00000059,0x0000005a,0x0000005b,0x000200f8,0x0000005a,0x0004003d,0x00000007,
0x0000005c,0x00000042,0x000500ba,0x00000056,0x0000005e,0x0000005c,0x0000005d,0x0004009a,
0x0000003b,0x0000005f,0x0000005e,0x000200f9,0x0000005b,0x000200f8,0x0000005b,0x000700f5,
0x0000003b,0x00000060,0x00000058,0x0000003d,0x0000005f,0x0000005a,0x000300f7,0x00000062,
0x00000000,0x000400fa,0x00000060,0x00000061,0x00000062,0x000200f8,0x00000061,0x000200fe,
0x00000063,0x000200f8,0x00000062,0x0004003d,0x00000066,0x00000069,0x00000068,0x00050041,
0x00000009,0x0000006a,0x00000042,0x00000024,0x0004003d,0x00000006,0x0000006b,0x0000006a,
0x0004003d,0x00000006,0x0000006c,0x00000012,0x00050081,0x00000006,0x0000006d,0x0000006b,
0x0000006c,0x00050085,0x00000006,0x0000006e,0x0000006d,0x0000003a,0x00050041,0x00000009,
0x0000006f,0x00000042,0x0000002b,0x0004003d,0x00000006,0x00000070,0x0000006f,0x00050050,
0x00000007,0x00000071,0x0000006e,0x00000070,0x00050057,0x0000001d,0x00000072,0x00000069,
0x00000071,0x0008004f,0x0000000f,0x00000073,0x00000072,0x00000072,0x00000000,0x00000001,
0x00000002,0x000200fe,0x00000073,0x000200f8,0x0000003e,0x00050041,0x00000079,0x0000007a,
0x00000020,0x00000078,0x0004003d,0x0000001d,0x0000007b,0x0000007a,0x0008004f,0x0000000f,
0x0000007c,0x0000007b,0x0000007b,0x00000000,0x00000001,0x00000002,0x00050041,0x00000079,
0x0000007e,0x00000020,0x0000007d,0x0004003d,0x0000001d,0x0000007f,0x0000007e,0x0008004f,
0x0000000f,0x00000080,0x0000007f,0x0000007f,0x00000000,0x00000001,0x00000002,0x00050041,
0x00000079,0x00000082,0x00000020,0x00000081,0x0004003d,0x0000001d,0x00000083,0x00000082,
0x0008004f,0x0000000f,0x00000084,0x00000083,0x00000083,0x00000000,0x00000001,0x00000002,
0x00050051,0x00000006,0x00000085,0x0000007c,0x00000000,0x00050051,0x00000006,0x00000086,
0x0000007c,0x00000001,0x00050051,0x00000006,0x00000087,0x0000007c,0x00000002,0x00050051,
0x00000006,0x00000088,0x00000080,0x00000000,0x00050051,0x00000006,0x00000089,0x00000080,
0x00000001,0x00050051,0x00000006,0x0000008a,0x00000080,0x00000002,0x00050051,0x00000006,
0x0000008b,0x00000084,0x00000000,0x00050051,0x00000006,0x0000008c,0x00000084,0x00000001,
0x00050051,0x00000006,0x0000008d,0x00000084,0x00000002,0x00060050,0x0000000f,0x0000008e,
0x00000085,0x00000086,0x00000087,0x00060050,0x0000000f,0x0000008f,0x00000088,0x00000089,
0x0000008a,0x00060050,0x0000000f,0x00000090,0x0000008b,0x0000008c,0x0000008d,0x00060050,
0x00000075,0x00000091,0x0000008e,0x0000008f,0x00000090,0x00040054,0x00000075,0x00000092,
0x00000091,0x0003003e,0x00000077,0x00000092,0x0004003d,0x00000075,0x00000095,0x00000077,
0x00050041,0x00000009,0x00000096,0x00000011,0x00000024,0x0004003d,0x00000006,0x00000097,
0x00000096,0x00050041,0x00000009,0x00000098,0x00000011,0x0000002b,0x0004003d,0x00000006,
0x00000099,0x00000098,0x0004007f,0x00000006,0x0000009a,0x00000099,0x00060050,0x0000000f,
0x0000009c,0x00000097,0x0000009a,0x0000009b,0x0006000c,0x0000000f,0x0000009d,0x00000001,
0x00000045,0x0000009c,0x00050091,0x0000000f,0x0000009e,0x00000095,0x0000009d,0x0003003e,
0x00000094,0x0000009e,0x0004003d,0x00000075,0x000000a0,0x00000077,0x0004003d,0x00000006,
0x000000a1,0x00000012,0x000500b8,0x0000003b,0x000000a2,0x000000a1,0x0000003a,0x000600a9,
0x00000006,0x000000a3,0x000000a2,0x0000009b,0x0000001c,0x00060041,0x00000025,0x000000a4,
0x00000020,0x00000078,0x00000046,0x0004003d,0x00000006,0x000000a5,0x000000a4,0x00050085,
0x00000006,0x000000a6,0x000000a3,0x000000a5,0x00060050,0x0000000f,0x000000a7,0x000000a6,
0x00000054,0x00000054,0x00050091,0x0000000f,0x000000a8,0x000000a0,0x000000a7,0x0003003e,
0x0000009f,0x000000a8,0x00050041,0x00000009,0x000000a9,0x00000094,0x00000037,0x0004003d,
0x00000006,0x000000aa,0x000000a9,0x000500ba,0x0000003b,0x000000ac,0x000000aa,0x000000ab,
0x000300f7,0x000000ae,0x00000000,0x000400fa,0x000000ac,0x000000ad,0x000000ae,0x000200f8,
0x000000ad,0x000200fe,0x000000b1,0x000200f8,0x000000ae,0x00060041,0x00000025,0x000000b4,
0x00000020,0x0000007d,0x00000046,0x0004003d,0x00000006,0x000000b5,0x000000b4,0x0004007f,
0x00000006,0x000000b6,0x000000b5,0x00050041,0x00000009,0x000000b7,0x0000009f,0x00000037,
0x0004003d,0x00000006,0x000000b8,0x000000b7,0x00050083,0x00000006,0x000000b9,0x000000b6,
0x000000b8,0x00050041,0x00000009,0x000000ba,0x00000094,0x00000037,0x0004003d,0x00000006,
0x000000bb,0x000000ba,0x00050088,0x00000006,0x000000bc,0x000000b9,0x000000bb,0x0003003e,
0x000000b3,0x000000bc,0x0004003d,0x0000000f,0x000000be,0x0000009f,0x0004003d,0x0000000f,
0x000000bf,0x00000094,0x0004003d,0x00000006,0x000000c0,0x000000b3,0x0005008e,0x0000000f,
0x000000c1,0x000000bf,0x000000c0,0x00050081,0x0000000f,0x000000c2,0x000000be,0x000000c1,
0x0003003e,0x000000bd,0x000000c2,0x00060041,0x00000025,0x000000c4,0x00000020,0x00000081,
0x00000046,0x0004003d,0x00000006,0x000000c5,0x000000c4,0x0003003e,0x000000c3,0x000000c5,
0x0004003d,0x00000006,0x000000c7,0x000000c3,0x00060041,0x00000025,0x000000c8,0x00000020,
0x00000036,0x0000002b,0x0004003d,0x00000006,0x000000c9,0x000000c8,0x00050088,0x00000006,
0x000000ca,0x000000c7,0x000000c9,0x0003003e,0x000000c6,0x000000ca,0x00050041,0x00000009,
0x000000cc,0x000000bd,0x00000024,0x0004003d,0x00000006,0x000000cd,0x000000cc,0x0004003d,
0x00000006,0x000000ce,0x000000c3,0x00050088,0x00000006,0x000000cf,0x000000cd,0x000000ce,
0x00050041,0x00000009,0x000000d0,0x000000bd,0x0000002b,0x0004003d,0x00000006,0x000000d1,
0x000000d0,0x0004007f,0x00000006,0x000000d2,0x000000d1,0x0004003d,0x00000006,0x000000d3,
0x000000c6,0x00050088,0x00000006,0x000000d4,0x000000d2,0x000000d3,0x00050050,0x00000007,
0x000000d5,0x000000cf,0x000000d4,0x0005008e,0x00000007,0x000000d6,0x000000d5,0x0000003a,
0x00050050,0x00000007,0x000000d7,0x0000003a,0x0000003a,0x00050081,0x00000007,0x000000d8,
0x000000d6,0x000000d7,0x0003003e,0x000000cb,0x000000d8,0x0004003d,0x00000007,0x000000d9,
0x000000cb,0x000500b8,0x00000056,0x000000da,0x000000d9,0x00000055,0x0004009a,0x0000003b,
0x000000db,0x000000da,0x000400a8,0x0000003b,0x000000dc,0x000000db,0x000300f7,0x000000de,
0x00000000,0x000400fa,0x000000dc,0x000000dd,0x000000de,0x000200f8,0x000000dd,0x0004003d,
0x00000007,0x000000df,0x000000cb,0x000500ba,0x00000056,0x000000e0,0x000000df,0x0000005d,
0x0004009a,0x0000003b,0x000000e1,0x000000e0,0x000200f9,0x000000de,0x000200f8,0x000000de,
0x000700f5,0x0000003b,0x000000e2,0x000000db,0x000000ae,0x000000e1,0x000000dd,0x000300f7,
0x000000e4,0x00000000,0x000400fa,0x000000e2,0x000000e3,0x000000e4,0x000200f8,0x000000e3,
0x000200fe,0x000000b1,0x000200f8,0x000000e4,0x0004003d,0x00000066,0x000000e6,0x00000068,
0x0004003d,0x00000007,0x000000e7,0x000000cb,0x00050057,0x0000001d,0x000000e8,0x000000e6,
0x000000e7,0x0008004f,0x0000000f,0x000000e9,0x000000e8,0x000000e8,0x00000000,0x00000001,
0x00000002,0x000200fe,0x000000e9,0x00010038
};
+56 -16
View File
@@ -6,6 +6,7 @@
#include <sys/stat.h>
#include <unistd.h>
#include <stdint.h>
#include <stdatomic.h>
#include <fcntl.h>
#include <time.h>
#include <stdarg.h>
@@ -271,7 +272,9 @@ volatile int g_ubwc = 1;
volatile int g_no_render_node;
volatile int g_output_refresh_mhz;
static atomic_int g_output_refresh_mhz = 60000;
static atomic_int g_output_refresh_pending;
static int g_output_announced_refresh_mhz = 60000;
volatile int g_output_w, g_output_h;
struct pending_release {
@@ -797,7 +800,7 @@ static void feedback_discard_all(struct wl_list *list) {
}
static uint32_t nominal_refresh_ns(void) {
int mhz = g_output_refresh_mhz;
int mhz = atomic_load(&g_output_refresh_mhz);
if (mhz <= 0) mhz = 60000;
return (uint32_t)(1000000000000LL / (int64_t)mhz);
}
@@ -1510,10 +1513,10 @@ static const struct wl_output_interface output_impl = { .release = output_releas
static struct wl_list g_outputs;
static void output_resource_destroy(struct wl_resource *r) { wl_list_remove(wl_resource_get_link(r)); }
static void output_send_mode(struct wl_resource *r) {
static void output_send_mode(struct wl_resource *r, int mhz) {
wl_output_send_mode(r, WL_OUTPUT_MODE_CURRENT | WL_OUTPUT_MODE_PREFERRED,
g_output_w > 0 ? g_output_w : 1920, g_output_h > 0 ? g_output_h : 1080,
g_output_refresh_mhz > 0 ? g_output_refresh_mhz : 60000);
mhz);
}
static void bind_output(struct wl_client *c, void *data, uint32_t ver, uint32_t id) {
@@ -1523,7 +1526,7 @@ static void bind_output(struct wl_client *c, void *data, uint32_t ver, uint32_t
wl_list_insert(&g_outputs, wl_resource_get_link(r));
wl_output_send_geometry(r, 0, 0, 340, 190, WL_OUTPUT_SUBPIXEL_UNKNOWN,
"DroidDeck", "Wayland", WL_OUTPUT_TRANSFORM_NORMAL);
output_send_mode(r);
output_send_mode(r, g_output_announced_refresh_mhz);
if (ver >= WL_OUTPUT_NAME_SINCE_VERSION) {
wl_output_send_name(r, "DroidDeck-1");
wl_output_send_description(r, "DroidDeck display");
@@ -2838,10 +2841,19 @@ static void deliver_key(const struct input_msg *m) {
key_event((uint32_t)m->p1, m->p2);
}
static void scroll_event(int steps) {
static void send_axis(struct seat_pointer *sp, uint32_t t, uint32_t axis, int value120) {
if (!value120) return;
if (wl_resource_get_version(sp->ptr) >= WL_POINTER_AXIS_VALUE120_SINCE_VERSION)
wl_pointer_send_axis_value120(sp->ptr, axis, value120);
else if (wl_resource_get_version(sp->ptr) >= WL_POINTER_AXIS_DISCRETE_SINCE_VERSION && value120 % 120 == 0)
wl_pointer_send_axis_discrete(sp->ptr, axis, value120 / 120);
wl_pointer_send_axis(sp->ptr, t, axis, wl_fixed_from_double(value120 / 12.0));
}
static void axis_event(int vertical120, int horizontal120) {
struct surface *target = g_active_constraint ? g_active_constraint->surface
: g_desktop ? g_desktop : (g_grab ? g_grab : toplevel_at(g_ptr_x, g_ptr_y));
if (!target || !steps) return;
if (!target || (!vertical120 && !horizontal120)) return;
struct wl_client *client = wl_resource_get_client(target->resource);
struct seat_pointer *sp;
if (!pointer_for(client)) return;
@@ -2850,17 +2862,18 @@ static void scroll_event(int steps) {
uint32_t t = now_ms();
pointer_focus(target->resource, wl_fixed_from_double(g_ptr_x - tx), wl_fixed_from_double(g_ptr_y - ty));
for_each_pointer_of(client, sp) {
if (wl_resource_get_version(sp->ptr) >= WL_POINTER_AXIS_VALUE120_SINCE_VERSION)
wl_pointer_send_axis_value120(sp->ptr, WL_POINTER_AXIS_VERTICAL_SCROLL, steps * 120);
else if (wl_resource_get_version(sp->ptr) >= WL_POINTER_AXIS_DISCRETE_SINCE_VERSION)
wl_pointer_send_axis_discrete(sp->ptr, WL_POINTER_AXIS_VERTICAL_SCROLL, steps);
wl_pointer_send_axis(sp->ptr, t, WL_POINTER_AXIS_VERTICAL_SCROLL, wl_fixed_from_int(steps * 10));
send_axis(sp, t, WL_POINTER_AXIS_VERTICAL_SCROLL, vertical120);
send_axis(sp, t, WL_POINTER_AXIS_HORIZONTAL_SCROLL, horizontal120);
if (wl_resource_get_version(sp->ptr) >= WL_POINTER_FRAME_SINCE_VERSION)
wl_pointer_send_frame(sp->ptr);
}
wl_display_flush_clients(g_display);
}
static void scroll_event(int steps) {
axis_event(steps * 120, 0);
}
static int update_modifiers(uint32_t evdev, int pressed) {
static const struct { uint32_t evdev, mask; } mod_keys[] = {
{ 42, 0x1 }, { 54, 0x1 },
@@ -2916,13 +2929,19 @@ static void key_event(uint32_t evdev, int pressed) {
wl_display_flush_clients(g_display);
}
static void output_resized(void) {
struct surface *s;
static void output_notify_mode(void) {
int mhz = atomic_load(&g_output_refresh_mhz);
struct wl_resource *r;
wl_resource_for_each(r, &g_outputs) {
output_send_mode(r);
output_send_mode(r, mhz);
if (wl_resource_get_version(r) >= 2) wl_output_send_done(r);
}
g_output_announced_refresh_mhz = mhz;
}
static void output_resized(void) {
struct surface *s;
output_notify_mode();
wl_list_for_each(s, &g_surfaces, link)
if (s->xdg_toplevel && s->toplevel_committed) send_toplevel_configure(s);
droiddeck_log("display", "output resized to %dx%d", g_output_w, g_output_h);
@@ -2931,13 +2950,15 @@ static void output_resized(void) {
static int on_input_readable(int fd, uint32_t mask, void *data) {
struct input_msg m;
int vsync = 0;
int64_t vsync_ns = 0;
while (read(fd, &m, sizeof(m)) == (ssize_t)sizeof(m)) {
switch (m.type) {
case 1: deliver_key(&m); break;
case 2: pointer_event(m.p1, m.p2, 0, 0); break;
case 3: pointer_button(m.p1, m.p2); break;
case 4: scroll_event(m.p1); break;
case 5: on_vsync(((int64_t)m.p1 << 32) | (uint32_t)m.p2); break;
case 5: vsync = 1; vsync_ns = ((int64_t)m.p1 << 32) | (uint32_t)m.p2; break;
case 6: pointer_delta(m.p1 / 256.0, m.p2 / 256.0); break;
case 7: deliver_touch(&m); break;
case 8: output_resized(); break;
@@ -2945,9 +2966,17 @@ static int on_input_readable(int fd, uint32_t mask, void *data) {
vk_present_set_scale_mode(m.p1, m.p2);
schedule_render();
break;
case 10: break;
case 11: axis_event(m.p1, m.p2); break;
default: deliver_pointer(&m); break;
}
}
if (atomic_exchange(&g_output_refresh_pending, 0) &&
atomic_load(&g_output_refresh_mhz) != g_output_announced_refresh_mhz) {
output_notify_mode();
wl_display_flush_clients(g_display);
}
if (vsync) on_vsync(vsync_ns);
return 0;
}
@@ -2989,6 +3018,16 @@ void droiddeck_wayland_output_resized(void) {
(void)n;
}
void droiddeck_wayland_set_output_refresh_rate(int mhz) {
if (mhz <= 0) mhz = 60000;
if (atomic_exchange(&g_output_refresh_mhz, mhz) == mhz || g_input_pipe[1] < 0) return;
atomic_store(&g_output_refresh_pending, 1);
struct input_msg m = { 10, 0, 0, 0, 0 };
ssize_t n;
do { n = write(g_input_pipe[1], &m, sizeof(m)); } while (n < 0 && errno == EINTR);
(void)n;
}
void droiddeck_wayland_vsync(int64_t frame_time_ns) {
if (g_input_pipe[1] < 0) return;
struct input_msg m = { 5, (int)(frame_time_ns >> 32), (int)(uint32_t)frame_time_ns, 0, 0 };
@@ -3226,6 +3265,7 @@ int droiddeck_wayland_run(void) {
if ((g_stats_timer = wl_event_loop_add_timer(wl_display_get_event_loop(display), on_stats_timer, NULL)))
wl_event_source_timer_update(g_stats_timer, 10000);
g_output_announced_refresh_mhz = atomic_load(&g_output_refresh_mhz);
wl_display_run(display);
wl_display_destroy(display);
@@ -9,6 +9,7 @@
enum {
DROIDDECK_HEADPOSE_LIVE = 1u << 0,
DROIDDECK_HEADPOSE_ENABLED = 1u << 1,
DROIDDECK_HEADPOSE_MONO = 1u << 2,
};
struct droiddeck_headpose {
@@ -22,7 +23,10 @@ struct droiddeck_headpose {
float tan_half_fov;
float eye_aspect;
uint32_t eye_w, eye_h;
uint32_t reserved[12];
float display_hz;
// Uses a formerly reserved word; older publishers leave this zero.
uint32_t render_height;
uint32_t reserved[10];
};
#ifdef __cplusplus
@@ -27,13 +27,13 @@ extern void droiddeck_wayland_send_key(int evdev, int state);
extern void droiddeck_wayland_send_scene_input(int type, int a, int b);
extern void droiddeck_wayland_vsync(int64_t frame_time_ns);
extern void droiddeck_wayland_output_resized(void);
extern void droiddeck_wayland_set_output_refresh_rate(int mhz);
extern volatile int g_fps_limit;
extern volatile int g_hide_shell;
extern volatile int g_zero_copy;
extern volatile unsigned g_zero_copy_last;
extern volatile int g_ubwc;
extern volatile int g_no_render_node;
extern volatile int g_output_refresh_mhz;
extern volatile int g_output_w, g_output_h;
#define TAG "DroidDeckWayland"
@@ -328,7 +328,7 @@ Java_com_droiddeck_launcher_wayland_WaylandCompositor_nativeSetLayerFrameRate(JN
JNIEXPORT void JNICALL
Java_com_droiddeck_launcher_wayland_WaylandCompositor_nativeSetOutputRefreshRate(JNIEnv *env, jclass clazz, jfloat hz) {
g_output_refresh_mhz = hz > 1.0f ? (int)(hz * 1000.0f + 0.5f) : 0;
droiddeck_wayland_set_output_refresh_rate(hz > 1.0f ? (int)(hz * 1000.0f + 0.5f) : 0);
}
extern void vkp_reset_first_frame(void);
@@ -342,6 +342,7 @@ JNIEXPORT void JNICALL
Java_com_droiddeck_launcher_wayland_WaylandCompositor_nativeSetOutputSize(JNIEnv *env, jclass clazz, jint w, jint h) {
g_output_w = w > 0 ? w : 0;
g_output_h = h > 0 ? h : 0;
vkp_cardboard_set_render_height(h);
}
JNIEXPORT void JNICALL
@@ -350,6 +351,7 @@ Java_com_droiddeck_launcher_wayland_WaylandCompositor_nativeResizeOutput(JNIEnv
if (w <= 0 || h <= 0) return;
g_output_w = w;
g_output_h = h;
vkp_cardboard_set_render_height(h);
droiddeck_wayland_output_resized();
}
@@ -402,10 +404,11 @@ Java_com_droiddeck_launcher_wayland_WaylandCompositor_nativeSetScreenEffects(JNI
JNIEXPORT void JNICALL
Java_com_droiddeck_launcher_wayland_WaylandCompositor_nativeSetCardboard(JNIEnv *env, jclass clazz, jboolean enabled,
jint source, jfloat ipd, jfloat dist, jfloat width, jfloat k1, jfloat k2, jfloat fov, jfloat inset, jfloat chroma) {
jint source, jfloat ipd, jfloat dist, jfloat width, jfloat k1, jfloat k2, jfloat fov, jfloat inset, jfloat chroma,
jboolean mono, jfloat hz) {
struct cardboard_params p = {.enabled = enabled ? 1 : 0, .source = source, .ipd_m = ipd, .screen_dist_m = dist,
.screen_width_m = width, .k1 = k1, .k2 = k2, .fov_deg = fov, .lens_inset = inset,
.chroma = chroma};
.chroma = chroma, .mono = mono ? 1 : 0, .display_hz = hz};
vkp_cardboard_set(&p);
}
@@ -16,12 +16,17 @@ class App : Application() {
override fun onCreate() {
super.onCreate()
// The restart screen must not initialize USB, guest processes, or another graphics context.
val process = (getSystemService(ACTIVITY_SERVICE) as android.app.ActivityManager)
.runningAppProcesses?.firstOrNull { it.pid == android.os.Process.myPid() }?.processName
if (process == "$packageName:vr_recovery") return
CrashHandler.install(this)
com.droiddeck.launcher.session.QuestExperimental.initialize(this)
com.droiddeck.launcher.session.SessionPrefs.settleGpuDriverMode(this)
com.droiddeck.launcher.update.AppUpdates.init(this)
com.droiddeck.launcher.core.DownloadSource.init(this)
GpuClockPin.clearLeftover(this)
com.droiddeck.launcher.glasses.Glasses.init(this)
BwrapSpawner.start(this)
com.droiddeck.launcher.frontend.GameFileSync.start(this)
com.droiddeck.launcher.stores.StoresState.init(this)
@@ -109,6 +109,10 @@ class MainActivity : ComponentActivity() {
private var frameGenLabel by mutableStateOf("")
private var lossless by mutableStateOf(Lossless.State.NONE)
private var showRemove by mutableStateOf(false)
private var pendingSwitch by mutableStateOf<Intent?>(null)
private var switchingTo: Intent? = null
private var switchPoll: Runnable? = null
private var switchDeadline = 0L
private var showResetSettings by mutableStateOf(false)
private var lowSpace by mutableStateOf<Long?>(null)
private var showNonAdreno by mutableStateOf<LinuxRuntimeInstaller.Release?>(null)
@@ -192,6 +196,10 @@ class MainActivity : ComponentActivity() {
SessionPrefs.setWifiDiscoveryEnabled(this, WifiDiscovery.permissionGranted(this))
refreshWifiDiscovery()
}
private val cameraPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()) {
com.droiddeck.launcher.handtracking.HandTracking.refreshPermission(this)
}
private var placementCheck by mutableStateOf<Boolean?>(null)
private val wifiPermissionSettings = registerForActivityResult(ActivityResultContracts.StartActivityForResult()) {
SessionPrefs.setWifiDiscoveryEnabled(this, WifiDiscovery.permissionGranted(this))
refreshWifiDiscovery()
@@ -363,6 +371,7 @@ class MainActivity : ComponentActivity() {
private var resolution by mutableStateOf(com.droiddeck.launcher.session.SessionDisplay.DEFAULT_RESOLUTION)
private var fexPreset by mutableStateOf("")
private var forceSsbs by mutableStateOf(false)
private var steamClientArch by mutableStateOf("arm64")
private var runSteamAtStartup by mutableStateOf(false)
private var bootMovie by mutableStateOf(com.droiddeck.launcher.session.BootMovie.DROIDDECK_30)
private var bootMovieSound by mutableStateOf(true)
@@ -391,11 +400,13 @@ class MainActivity : ComponentActivity() {
}
private var romsDir by mutableStateOf<String?>(null)
private var steamGames by mutableStateOf<List<Library.SteamGame>>(emptyList())
private var steamGamesX64 by mutableStateOf(false)
@Volatile private var componentSweepDone = false
@Volatile private var componentSweepSession: String? = null
private var emulatorList by mutableStateOf<List<Library.Emulator>>(emptyList())
private var runningLabel by mutableStateOf<String?>(null)
private var logsEnabled by mutableStateOf(true)
private var vacDiagnostics by mutableStateOf(false)
private var agentCommands by mutableStateOf(false)
private var showRoms by mutableStateOf(false)
private var homeAppSelected by mutableStateOf(false)
@@ -502,24 +513,186 @@ class MainActivity : ComponentActivity() {
android.widget.Toast.makeText(this, R.string.main_wait_install_game, android.widget.Toast.LENGTH_SHORT).show()
return false
}
if (com.droiddeck.launcher.session.NoSteamLaunch.enabled(this, game)) return launchWithoutSteam(game, returnHome)
if (SessionState.running) {
if (SessionState.mode != SessionService.MODE_STEAM || SessionState.stopRequested) {
android.widget.Toast.makeText(this, R.string.game_link_session_busy, android.widget.Toast.LENGTH_LONG).show()
return true
}
if (!SessionService.launchGame(this, game.gameIdString)) {
val steamVr = game.gameIdString == com.droiddeck.launcher.gpu.SteamVr.APP_ID
if (!steamVr && !SessionService.launchGame(this, game.gameIdString)) {
android.widget.Toast.makeText(this, R.string.game_link_failed, android.widget.Toast.LENGTH_LONG).show()
return false
}
startActivity(Intent(this, SessionActivity::class.java).setAction(SessionService.ACTION_RESUME)
.putExtra(SessionActivity.EXTRA_STEAMVR, steamVr)
.putExtra(SessionActivity.EXTRA_RETURN_HOME, returnHome)
.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP or Intent.FLAG_ACTIVITY_SINGLE_TOP))
return true
}
return startSession(Intent(this, SessionActivity::class.java)
.putExtra(SessionActivity.EXTRA_RETURN_HOME, returnHome)
.putExtra(SessionService.EXTRA_STEAM_URL,
com.droiddeck.launcher.frontend.GameLaunchLink.steamUrl(game.gameIdString)), steamSession = true)
.apply {
if (game.gameIdString == com.droiddeck.launcher.gpu.SteamVr.APP_ID) {
putExtra(SessionActivity.EXTRA_STEAMVR, true)
} else {
putExtra(SessionService.EXTRA_STEAM_URL,
com.droiddeck.launcher.frontend.GameLaunchLink.steamUrl(game.gameIdString))
}
}, steamSession = true)
}
private fun launchWithoutSteam(game: Library.SteamGame, returnHome: Boolean): Boolean {
val app = applicationContext
Thread({
val added = if (game.library == Library.ADDED) runCatching { com.droiddeck.launcher.session.NoSteamLaunch.addedGame(app, game) }.getOrNull() else null
val command = com.droiddeck.launcher.session.NoSteamLaunch.command(game, added)
ui.post {
if (isDestroyed) return@post
if (command == null) {
android.widget.Toast.makeText(this, R.string.game_link_failed, android.widget.Toast.LENGTH_LONG).show()
return@post
}
com.droiddeck.launcher.session.NoSteamLaunch.remember(command)
val next = Intent(this, SessionActivity::class.java)
.putExtra(SessionActivity.EXTRA_RETURN_HOME, returnHome)
.putExtra(SessionService.EXTRA_MODE, SessionService.MODE_RUN)
.putExtra(SessionService.EXTRA_PROGRAM, command.program)
.putExtra(SessionService.EXTRA_PROGRAM_ARGS, command.args.toTypedArray())
when {
!SessionState.running -> startSessionOrWait(next)
SessionState.stopRequested -> android.widget.Toast.makeText(this, R.string.game_link_session_busy, android.widget.Toast.LENGTH_LONG).show()
SessionState.mode == SessionService.MODE_RUN && SessionState.program == command.program && SessionState.programArgs == command.args ->
startActivity(Intent(this, SessionActivity::class.java).setAction(SessionService.ACTION_RESUME)
.putExtra(SessionActivity.EXTRA_RETURN_HOME, returnHome)
.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP or Intent.FLAG_ACTIVITY_SINGLE_TOP))
else -> pendingSwitch = next
}
}
}, "no-steam-launch").start()
return true
}
private fun opensEpicLauncher(next: Intent): Boolean =
next.getStringExtra(SessionService.EXTRA_PROGRAM) == com.droiddeck.launcher.session.NoSteamLaunch.EPIC ||
next.getStringExtra(SessionService.EXTRA_STEAM_URL) == com.droiddeck.launcher.frontend.GameLaunchLink.steamUrl(
com.droiddeck.launcher.stores.epic.EpicClient.gameId(com.droiddeck.launcher.stores.epic.EpicClient.shortcutAppId))
private fun uninstallEpicLauncher() {
if (com.droiddeck.launcher.stores.epic.EpicClient.uninstalling) return
com.droiddeck.launcher.stores.epic.EpicClient.uninstalling = true
pendingSwitch?.takeIf(::opensEpicLauncher)?.let { pendingSwitch = null }
switchingTo?.takeIf(::opensEpicLauncher)?.let {
switchPoll?.let { poll -> ui.removeCallbacks(poll) }
switchPoll = null
switchingTo = null
}
if (SessionState.running) SessionService.stop(this)
val app = applicationContext
val deadline = android.os.SystemClock.uptimeMillis() + SESSION_SWITCH_WAIT_MS
Thread({
while ((SessionState.running || SessionState.phase !in SESSION_ENDED) && android.os.SystemClock.uptimeMillis() < deadline) {
Thread.sleep(SESSION_SWITCH_POLL_MS)
}
val ended = !SessionState.running && SessionState.phase in SESSION_ENDED
val done = try {
ended && com.droiddeck.launcher.stores.epic.EpicClient.uninstall(app)
} finally {
com.droiddeck.launcher.stores.epic.EpicClient.uninstalling = false
}
ui.post {
if (!isDestroyed) refresh()
android.widget.Toast.makeText(app, if (done) R.string.epic_client_uninstalled else R.string.epic_client_uninstall_failed,
android.widget.Toast.LENGTH_LONG).show()
}
}, "epic-uninstall").start()
}
private fun openEpicLauncher(): Boolean {
if (com.droiddeck.launcher.stores.epic.EpicClient.uninstalling) {
android.widget.Toast.makeText(this, R.string.epic_client_uninstalling, android.widget.Toast.LENGTH_SHORT).show()
return false
}
if (protons.protonBusyId != null || ProtonExtras.installInProgress || pkgStage != null) {
android.widget.Toast.makeText(this, R.string.main_wait_install_game, android.widget.Toast.LENGTH_SHORT).show()
return false
}
if (!com.droiddeck.launcher.stores.epic.EpicClient.launcherRunning(this) && !com.droiddeck.launcher.stores.epic.EpicClient.fexReady(this)) {
com.droiddeck.launcher.stores.epic.EpicClient.requestFex(this)
android.widget.Toast.makeText(this, R.string.epic_client_needs_fex, android.widget.Toast.LENGTH_LONG).show()
if (!SessionState.running) startSteamSession()
else if (SessionState.mode == SessionService.MODE_STEAM) startActivity(Intent(this, SessionActivity::class.java)
.setAction(SessionService.ACTION_RESUME).addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP or Intent.FLAG_ACTIVITY_SINGLE_TOP))
return true
}
if (com.droiddeck.launcher.session.NoSteamLaunch.enabled(this, com.droiddeck.launcher.stores.epic.EpicClient.shortcutAppId)) {
com.droiddeck.launcher.stores.epic.EpicClient.launcherEntry(this, Library.EPIC_LAUNCHER_NAME)?.let { return launchWithoutSteam(it, false) }
}
val launcher = Intent(this, SessionActivity::class.java)
.putExtra(SessionActivity.EXTRA_RETURN_HOME, false)
.putExtra(SessionService.EXTRA_STEAM_URL, com.droiddeck.launcher.frontend.GameLaunchLink.steamUrl(
com.droiddeck.launcher.stores.epic.EpicClient.gameId(com.droiddeck.launcher.stores.epic.EpicClient.shortcutAppId)))
if (SessionState.running && SessionState.mode == SessionService.MODE_STEAM && !SessionState.stopRequested) {
val app = applicationContext
val shortcut = com.droiddeck.launcher.stores.epic.EpicClient.shortcutAppId
val listed = com.droiddeck.launcher.frontend.AddedGames.inSteam(app, shortcut)
if (listed && !SessionService.launchGame(this, com.droiddeck.launcher.stores.epic.EpicClient.gameId(shortcut))) {
android.widget.Toast.makeText(this, R.string.game_link_failed, android.widget.Toast.LENGTH_LONG).show()
return false
}
startActivity(Intent(this, SessionActivity::class.java).setAction(SessionService.ACTION_RESUME)
.putExtra(SessionActivity.EXTRA_RETURN_HOME, false)
.addFlags(Intent.FLAG_ACTIVITY_CLEAR_TOP or Intent.FLAG_ACTIVITY_SINGLE_TOP))
if (!listed) Thread({
com.droiddeck.launcher.frontend.AddedGamesPublisher.republish(app)
val id = com.droiddeck.launcher.stores.epic.EpicClient.steamLauncherId(app) ?: shortcut
ui.post { SessionService.launchGame(app, com.droiddeck.launcher.stores.epic.EpicClient.gameId(id)) }
}, "epic-launcher-open").start()
return true
}
if (SessionState.running) {
if (SessionState.stopRequested) android.widget.Toast.makeText(this, R.string.game_link_session_busy, android.widget.Toast.LENGTH_LONG).show()
else pendingSwitch = launcher
return true
}
return startSessionOrWait(launcher)
}
private fun startSessionOrWait(next: Intent): Boolean {
val started = startSession(next, steamSession = next.hasExtra(SessionService.EXTRA_STEAM_URL))
if (!started && (busy || LinuxRuntimeInstaller.isBusy())) android.widget.Toast.makeText(this, R.string.main_wait_install_game, android.widget.Toast.LENGTH_SHORT).show()
return started
}
private fun startAfterSession(next: Intent) {
SessionService.stop(this)
awaitSessionEnd(next, android.os.SystemClock.uptimeMillis() + SESSION_SWITCH_WAIT_MS)
}
private fun awaitSessionEnd(next: Intent, deadline: Long) {
switchingTo = next
switchDeadline = deadline
switchPoll?.let { ui.removeCallbacks(it) }
val poll = object : Runnable {
override fun run() {
if (isDestroyed) return
if (SessionState.running || SessionState.phase !in SESSION_ENDED) {
if (android.os.SystemClock.uptimeMillis() < deadline) ui.postDelayed(this, SESSION_SWITCH_POLL_MS)
else {
switchPoll = null
switchingTo = null
android.widget.Toast.makeText(this@MainActivity, R.string.game_link_session_busy, android.widget.Toast.LENGTH_LONG).show()
}
return
}
switchPoll = null
switchingTo = null
if (com.droiddeck.launcher.stores.epic.EpicClient.uninstalling && opensEpicLauncher(next)) return
startSessionOrWait(next)
}
}
switchPoll = poll
ui.postDelayed(poll, SESSION_SWITCH_POLL_MS)
}
private fun handleGameLink(games: List<Library.SteamGame>) {
@@ -554,7 +727,9 @@ class MainActivity : ComponentActivity() {
theme = SessionPrefs.theme(this)
frameGenLabel = getString(R.string.frame_gen_off)
appScale = com.droiddeck.launcher.core.AppUiPrefs.scale(this)
steamGames = if (shortcutPicker) emptyList() else com.droiddeck.launcher.frontend.LibraryCache.load(this)
val (cached, cachedX64) = com.droiddeck.launcher.frontend.LibraryCache.snapshot(this)
steamGames = if (shortcutPicker) emptyList() else cached
steamGamesX64 = cachedX64
backActionsInverted = SessionPrefs.backActionsInverted(this)
orientation = SessionPrefs.orientation(this)
requestedOrientation = orientation
@@ -591,8 +766,9 @@ class MainActivity : ComponentActivity() {
busy = busy, stage = stage, percent = percent,
desktopInstalled = desktopInstalled, desktopPresent = desktopPresent,
offlineAccount = offlineAccount, offline = offline,
frameGenLabel = frameGenLabel, romsDir = romsDir, logsEnabled = logsEnabled, agentCommands = agentCommands,
steamGames = steamGames, emulators = emulatorList, running = runningLabel,
frameGenLabel = frameGenLabel, romsDir = romsDir, logsEnabled = logsEnabled,
vacDiagnostics = vacDiagnostics, agentCommands = agentCommands,
steamGames = steamGames, steamGamesX64 = steamGamesX64, emulators = emulatorList, running = runningLabel,
frameGen = FrameGen.mode(this),
lossless = lossless,
pageKey = if (showPerformance) "performance" else if (showComponents) "components" else if (showMapping) "controller-mapping" else null,
@@ -636,6 +812,7 @@ class MainActivity : ComponentActivity() {
),
FrontEndActions(
onPlay = { playSteam() },
onPlaySteamVr = { playSteamVr() },
onSteamUrl = { url ->
if (!SessionState.running) startSession(
Intent(this, SessionActivity::class.java).putExtra(SessionService.EXTRA_STEAM_URL, url), steamSession = true,
@@ -653,6 +830,8 @@ class MainActivity : ComponentActivity() {
com.droiddeck.launcher.stores.StoreLaunch.prepare(this, g) { launchGame(g) }
}
},
onEpicLauncher = { openEpicLauncher() },
onUninstallEpicLauncher = { uninstallEpicLauncher() },
onGameShortcut = { g -> com.droiddeck.launcher.frontend.GameShortcuts.pin(this, g) },
onExportGameFile = { g -> pickGameExport(g) },
onSyncGameFiles = { pickGameExport(null) },
@@ -738,6 +917,10 @@ class MainActivity : ComponentActivity() {
SessionPrefs.setLogsEnabled(this, !SessionPrefs.logsEnabled(this))
logsEnabled = SessionPrefs.logsEnabled(this)
},
onVacDiagnostics = {
SessionPrefs.setVacDiagnostics(this, !SessionPrefs.vacDiagnostics(this))
vacDiagnostics = SessionPrefs.vacDiagnostics(this)
},
onAgentCommands = {
com.droiddeck.launcher.agent.AgentAccess.setCommandsEnabled(this, !com.droiddeck.launcher.agent.AgentAccess.commandsEnabled(this))
agentCommands = com.droiddeck.launcher.agent.AgentAccess.commandsEnabled(this)
@@ -886,6 +1069,7 @@ class MainActivity : ComponentActivity() {
onQamButton = { on -> ControllerPrefs.setQamButton(this, on); refreshController() },
onKeyboardButton = { on -> ControllerPrefs.setKeyboardButton(this, on); refreshController() },
onSelectSteam = { on -> ControllerPrefs.setSelectSteam(this, on); refreshController() },
onJoyConVr = { on -> ControllerPrefs.setJoyConVr(this, on); refreshController() },
onEditLayout = { startActivity(Intent(this, ControllerEditorActivity::class.java)) },
onResetLayout = { ControllerPrefs.resetAllLayouts(this); refreshController() },
onMapping = { showPerformance = false; showComponents = false; showMapping = true },
@@ -913,6 +1097,15 @@ class MainActivity : ComponentActivity() {
onDismiss = { showNonAdreno = null },
)
}
pendingSwitch?.let { next ->
ConfirmDialog(
title = androidx.compose.ui.res.stringResource(R.string.drawer_stop_title),
text = runningLabel ?: "",
confirm = androidx.compose.ui.res.stringResource(R.string.drawer_stop),
onConfirm = { pendingSwitch = null; startAfterSession(next) },
onDismiss = { pendingSwitch = null },
)
}
if (showRemove) ConfirmDialog(
title = androidx.compose.ui.res.stringResource(R.string.main_remove_runtime_title),
text = androidx.compose.ui.res.stringResource(R.string.main_remove_runtime_text),
@@ -928,6 +1121,12 @@ class MainActivity : ComponentActivity() {
onDismiss = { lowSpace = null },
)
}
com.droiddeck.launcher.ui.HandCameraHost(
check = placementCheck,
onAskPermission = { cameraPermission.launch(Manifest.permission.CAMERA) },
onPlay = { placementCheck = null; startSteamSession(steamVr = true) },
onClose = { placementCheck = null },
)
if (showResetSettings) com.droiddeck.launcher.ui.ResetSettingsDialog(
onConfirm = { gameProfiles -> resetSettings(gameProfiles) },
onDismiss = { showResetSettings = false },
@@ -945,6 +1144,10 @@ class MainActivity : ComponentActivity() {
}
if (savedInstanceState == null) ui.post { startSteamAtStartupIfEnabled() }
else {
pendingSwitch = savedIntent(savedInstanceState, KEY_PENDING_SWITCH)
savedIntent(savedInstanceState, KEY_SWITCHING_TO)?.let { awaitSessionEnd(it, savedInstanceState.getLong(KEY_SWITCH_DEADLINE)) }
}
val wanted = ArrayList<String>()
if (checkSelfPermission(Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) {
@@ -988,8 +1191,13 @@ class MainActivity : ComponentActivity() {
} else startSteamSession()
}
private fun startSteamSession() {
startSession(Intent(this, SessionActivity::class.java), steamSession = true)
private fun playSteamVr() {
if (com.droiddeck.launcher.handtracking.HandTracking.wanted(this)) placementCheck = true
else startSteamSession(steamVr = true)
}
private fun startSteamSession(steamVr: Boolean = false) {
startSession(Intent(this, SessionActivity::class.java).putExtra(SessionActivity.EXTRA_STEAMVR, steamVr), steamSession = true)
}
override fun onWindowFocusChanged(hasFocus: Boolean) {
@@ -1086,6 +1294,19 @@ class MainActivity : ComponentActivity() {
refreshSecondScreenDisplays()
}
override fun onSaveInstanceState(outState: Bundle) {
super.onSaveInstanceState(outState)
pendingSwitch?.let { outState.putParcelable(KEY_PENDING_SWITCH, it) }
switchingTo?.let {
outState.putParcelable(KEY_SWITCHING_TO, it)
outState.putLong(KEY_SWITCH_DEADLINE, switchDeadline)
}
}
private fun savedIntent(state: Bundle, key: String): Intent? =
if (Build.VERSION.SDK_INT >= 33) state.getParcelable(key, Intent::class.java)
else @Suppress("DEPRECATION") state.getParcelable(key)
override fun onDestroy() {
updates.unregister()
super.onDestroy()
@@ -1354,6 +1575,7 @@ class MainActivity : ComponentActivity() {
fexPreset = if (mode == SessionService.MODE_STEAM) fexPreset else null,
forceSsbs = forceSsbs,
syncBackend = if (mode == SessionService.MODE_STEAM) SessionPrefs.syncBackendOf(fastSync, fsyncFirst, syncFallback) else null,
steamClientArch = if (mode == SessionService.MODE_STEAM) steamClientArch else null,
steamRepairQueued = steamRepairQueued,
mangoapp = mangoapp,
steamController = if (mode == SessionService.MODE_STEAM) steamController else null,
@@ -1437,6 +1659,7 @@ class MainActivity : ComponentActivity() {
syncFallback = SessionPrefs.syncFallback(this)
},
onSteamRepair = { steamRepairQueued = SteamRepair.queue(this) },
onSteamClientArch = { id -> SessionPrefs.setSteamClientArch(this, id); steamClientArch = id; refresh() },
onMangoapp = { on -> SessionPrefs.setMangoapp(this, on); mangoapp = on },
onSteamController = { id -> SessionPrefs.setSteamController(this, id); steamController = id },
onRunSteamAtStartup = { on ->
@@ -1576,6 +1799,7 @@ class MainActivity : ComponentActivity() {
fastSync = SessionPrefs.fastSync(this)
fsyncFirst = SessionPrefs.fsyncFirst(this)
syncFallback = SessionPrefs.syncFallback(this)
steamClientArch = SessionPrefs.steamClientArch(this)
steamRepairQueued = SteamRepair.queued(this)
mangoapp = SessionPrefs.mangoapp(this)
steamController = SessionPrefs.steamController(this)
@@ -1652,6 +1876,12 @@ class MainActivity : ComponentActivity() {
}
private fun refresh() {
// Until the scan finishes, the other client's games would launch in this one.
val listX64 = SessionPrefs.activeSteamClientArch(this) == "x86_64"
if (listX64 != steamGamesX64) {
steamGamesX64 = listX64
steamGames = steamGames.filterNot(Library::fromSteamClient)
}
if (!busy && LinuxRuntimeInstaller.isBusy()) followRuntimeOperation { LinuxRuntimeInstaller.attach(it) }
if (showComponents) drivers.refreshDrivers()
addedGamesArt = SessionPrefs.addedGamesArt(this)
@@ -1665,6 +1895,7 @@ class MainActivity : ComponentActivity() {
frameGenLabel = FrameGen.label(this)
romsDir = SessionPrefs.romsDir(this).takeIf { it.isNotEmpty() }
logsEnabled = SessionPrefs.logsEnabled(this)
vacDiagnostics = SessionPrefs.vacDiagnostics(this)
agentCommands = com.droiddeck.launcher.agent.AgentAccess.commandsEnabled(this)
runningLabel = if (SessionState.running) when (SessionState.mode) {
SessionService.MODE_DESKTOP -> getString(R.string.rail_desktop)
@@ -1675,6 +1906,9 @@ class MainActivity : ComponentActivity() {
val scanGeneration = ++libraryScanGeneration
val artRefreshesAtStart = HashMap(steamArtRefreshes)
if (shortcutPicker) shortcutLibraryScanning = true
if (SessionPrefs.steamClientArch(this) == "x86_64" && !OfflineMode.enabled(this)) {
Thread({ com.droiddeck.launcher.session.FexCatalog.refresh(this) }, "fex-catalog").start()
}
Thread({
val emus = Library.emulators(this) { id -> DesktopCatalog.installed(this, id) != null }
ui.post { if (scanGeneration == libraryScanGeneration) emulatorList = emus }
@@ -1691,9 +1925,11 @@ class MainActivity : ComponentActivity() {
}
}
}
val games = if (ready) Library.launchableGames(this) else emptyList()
val scannedArch = SessionPrefs.activeSteamClientArch(this)
val scannedX64 = scannedArch == "x86_64"
val games = if (ready) Library.launchableGames(this, arch = scannedArch) else emptyList()
val all = games.distinctBy { it.gameId }
if (ready) com.droiddeck.launcher.frontend.LibraryCache.save(this, all)
if (ready && scanGeneration == libraryScanGeneration) com.droiddeck.launcher.frontend.LibraryCache.save(this, all, scannedX64)
val endedSession = SessionState.sessionId.takeIf { !SessionState.running && it != null && it != componentSweepSession }
if (ready && (!componentSweepDone || endedSession != null)) {
componentSweepDone = true
@@ -1703,6 +1939,7 @@ class MainActivity : ComponentActivity() {
ui.post {
if (scanGeneration == libraryScanGeneration) {
val current = steamGames.filter { it.library != Library.ADDED }.associateBy { it.appId }
steamGamesX64 = scannedX64
steamGames = all.map { g ->
val newer = current[g.appId]
if (newer != null && g.library != Library.ADDED && steamArtRefreshes[g.appId] != artRefreshesAtStart[g.appId]) g.withArt(newer.art, newer.hero, newer.icon) else g
@@ -1852,6 +2089,12 @@ class MainActivity : ComponentActivity() {
companion object {
private const val TAG = "MainActivity"
private const val MEDIA_SETTLE_MS = 1500L
private const val SESSION_SWITCH_WAIT_MS = 30_000L
private const val SESSION_SWITCH_POLL_MS = 250L
private const val KEY_PENDING_SWITCH = "pending_switch"
private const val KEY_SWITCHING_TO = "switching_to"
private const val KEY_SWITCH_DEADLINE = "switch_deadline"
private val SESSION_ENDED = setOf(SessionPhase.IDLE, SessionPhase.FAILED)
private val ZIP_EXT = listOf("zip")
private val WCP_EXT = listOf("wcp")
}
@@ -26,7 +26,7 @@ class QuestXrActivity : NativeActivity() {
if (active) {
check(CompositorHost.isStarted) { "DroidDeck compositor is not running" }
SessionService.resume(this)
WaylandCompositor.nativeSetCardboard(false, 1, 0.064f, 3f, 4f, 0f, 0f, 90f, 0f, 0f)
WaylandCompositor.nativeSetCardboard(false, 1, 0.064f, 3f, 4f, 0f, 0f, 90f, 0f, 0f, false, 0f)
WaylandCompositor.nativeSetFrameGenArmed(false, 0, 0)
WaylandCompositor.nativeSetZeroCopy(false)
CompositorHost.attachXr()
@@ -48,10 +48,16 @@ import com.droiddeck.launcher.stores.Store
import com.droiddeck.launcher.stores.StoreAchievements
import com.droiddeck.launcher.stores.StoreGameSidecar
import com.droiddeck.launcher.stores.StoreLaunchRequests
import com.droiddeck.launcher.stores.epic.EpicClient
import com.droiddeck.launcher.ui.SessionAchievements
import com.droiddeck.launcher.gpu.Cardboard
import com.droiddeck.launcher.gpu.FrameGen
import com.droiddeck.launcher.gpu.HeadTracker
import com.droiddeck.launcher.glasses.Glasses
import com.droiddeck.launcher.glasses.GlassesScreen
import com.droiddeck.launcher.glasses.GlassesRendering
import kotlinx.coroutines.cancel
import kotlinx.coroutines.launch
import com.droiddeck.launcher.gpu.SteamVr
import com.droiddeck.launcher.gpu.ScreenEffectLooks
import com.droiddeck.launcher.gpu.ScreenEffects
@@ -64,12 +70,15 @@ import com.droiddeck.launcher.input.ControllerPrefs
import com.droiddeck.launcher.input.SessionClipboard
import com.droiddeck.launcher.input.OnScreenControls
import com.droiddeck.launcher.input.PadBridge
import com.droiddeck.launcher.input.PadMouse
import com.droiddeck.launcher.input.PadState
import com.droiddeck.launcher.input.SteamControllerBle
import com.droiddeck.launcher.input.PointerGestures
import com.droiddeck.launcher.input.SecondScreenDisplay
import com.droiddeck.launcher.input.SecondScreenDisplays
import com.droiddeck.launcher.input.SecondScreenMode
import com.droiddeck.launcher.input.TouchpadGestures
import com.droiddeck.launcher.input.ScrollTouchGestures
import com.droiddeck.launcher.runtime.LinuxRuntime
import com.droiddeck.launcher.session.LoadingState
import com.droiddeck.launcher.session.PerfHud
@@ -141,6 +150,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
private lateinit var loading: LoadingState
private lateinit var hud: PerfHud
private var padBridge: PadBridge? = null
private var joyConVr: com.droiddeck.launcher.input.JoyConVr? = null
private var handCamera: com.droiddeck.launcher.handtracking.HandCamera? = null
private var padMotion: com.droiddeck.launcher.input.PadMotion? = null
private var resumed = false
private val deckPadListener: () -> Unit = { uiHandler.post { updatePadMotion(); openDeckRing("deck pad changed"); applyUnfoldedControls() } }
@@ -176,6 +187,13 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
private var watching = true
private lateinit var touchpad: TouchpadGestures
private lateinit var threeFingerTap: com.droiddeck.launcher.input.ThreeFingerTap
private lateinit var scrollTouch: ScrollTouchGestures
private lateinit var padMouse: PadMouse
@Volatile private var padMouseOn = false
private var padMouseCheckedAt = 0L
private var scrollTouchGesture = false
private var scrollRestX = 0f
private var scrollRestY = 0f
private var touchMode by mutableStateOf(SessionPrefs.TOUCH_AUTO)
private var cursorPos by mutableStateOf(androidx.compose.ui.geometry.Offset(-100f, -100f))
private var cursorVisible by mutableStateOf(false)
@@ -191,6 +209,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
if (pointerLocked == locked) return@post
pointerLocked = locked
Log.i(TAG, "pointer lock " + if (locked) "taken by a program" else "released")
padMouseCheckedAt = 0L
if (::padMouse.isInitialized) padMouseMode()
// Touchpad deltas are measured from the middle while locked (moveLockedPointer).
if (locked && ::touchpad.isInitialized) {
drawnRect()?.let { touchpad.place(it.centerX(), it.centerY()) }
@@ -278,6 +298,23 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
private var steamVrStatus by mutableStateOf<SteamVr.Status?>(null)
private var steamVrSwitchedPicture by mutableStateOf(false)
private var steamVrSeenRunning = false
private val steamVrRecovery by lazy { com.droiddeck.launcher.gpu.SteamVrRecovery(this) {
resumed && !ended && SessionState.running && !isInPictureInPictureMode
} }
private var activeCardboard = Cardboard.OFF
private var glasses by mutableStateOf(Glasses.state.value)
private var glassesSettings by mutableStateOf(Glasses.settings.value)
private var glassesScreen: GlassesScreen? = null
private var glassesOutputActive by mutableStateOf(false)
private var glasses3dForVr = false
private var ended = false
private var glassesPresent = false
private var steamVrAutoTried = false
private var steamVrInstallTried = false
private var headTracking by mutableStateOf(false)
private var glassesSlot by mutableStateOf(0)
private var glassesKey: List<Any?> = emptyList()
private val glassesScope = kotlinx.coroutines.MainScope()
private val hasRotationSensor by lazy {
(getSystemService(SENSOR_SERVICE) as android.hardware.SensorManager).let {
it.getDefaultSensor(android.hardware.Sensor.TYPE_GAME_ROTATION_VECTOR)
@@ -318,10 +355,10 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
private fun followPanelRefresh(displayId: Int) {
if (!CompositorHost.isStarted) return
val display = (if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay) ?: return
if (display.displayId != displayId) return
val hz = display.refreshRate
if (hz > 1f) WaylandCompositor.nativeSetOutputRefreshRate(hz)
val output = glassesScreen?.display
?: (if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay) ?: return
if (output.displayId != displayId) return
applyOutputPacing()
}
private var isHomeApp by mutableStateOf(false)
private var androidApps by mutableStateOf<List<HomeApp.LaunchableApp>>(emptyList())
@@ -329,17 +366,25 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
private val deviceListener = object : InputManager.InputDeviceListener {
override fun onInputDeviceAdded(deviceId: Int) {
logInputDevice("connected", deviceId)
joyConVr?.refresh()
padBridge?.syncConnectedControllers()
refreshControls()
updatePointerCapture()
updateHandCamera()
}
override fun onInputDeviceRemoved(deviceId: Int) {
Log.i(TAG, "input device $deviceId disconnected")
joyConVr?.refresh()
padBridge?.onDeviceRemoved(deviceId)
if (::padMouse.isInitialized) padMouse.reset()
refreshControls()
updatePointerCapture()
updateHandCamera()
}
override fun onInputDeviceChanged(deviceId: Int) {
joyConVr?.refresh()
refreshControls()
}
override fun onInputDeviceChanged(deviceId: Int) = refreshControls()
}
private var runtimeRemovalBlocked = false
@@ -363,17 +408,21 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
SessionState.programArgs = intent.getStringArrayExtra(SessionService.EXTRA_PROGRAM_ARGS)?.toList().orEmpty()
SessionState.steamUi = intent.getStringExtra(SessionService.EXTRA_STEAM_UI)
SessionState.steamUrl = intent.getStringExtra(SessionService.EXTRA_STEAM_URL)
SessionState.steamVr = intent.getBooleanExtra(EXTRA_STEAMVR, false)
} else if (intent.getBooleanExtra(EXTRA_STEAMVR, false) && SessionState.mode == SessionService.MODE_STEAM) {
SessionState.steamVr = true
}
if (intent.action == SessionService.ACTION_AGENT_START) {
SessionEvents.record("agent.start_requested", mapOf("mode" to SessionState.mode))
}
com.droiddeck.launcher.gpu.SteamVrHealth.begin(this, SessionState.sessionId)
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
if (!SessionState.running) {
SessionState.fpsLimit = SessionPrefs.fpsLimit(
this, SessionPrefs.prefMode(intent.getStringExtra(SessionService.EXTRA_MODE) ?: SessionService.MODE_STEAM),
)
}
Log.i(TAG, "perf: " + PerfMode.apply(this, SessionState.fpsLimit))
applyOutputPacing()
if (!SessionState.running) SessionState.followScreen = followChoice()
applyFollowOrientation(resources.configuration)
goFullscreen()
@@ -403,6 +452,16 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
bridge.setSelectSteam(ControllerPrefs.read(this).selectSteam)
padBridge = bridge
SessionState.padBridge = bridge
if (Build.VERSION.SDK_INT >= 31) {
val vr = com.droiddeck.launcher.input.JoyConVr(this, File(filesDir, ".wayland-rt/${com.droiddeck.launcher.input.JoyConVr.FILE}")) { taken, released ->
if (joyConVr == null) return@JoyConVr
taken.forEach(bridge::onDeviceRemoved)
if (released.isNotEmpty()) bridge.syncConnectedControllers()
updateHandCamera()
}
bridge.setExcluded(vr::owns)
joyConVr = vr
}
SteamControllerBle.listener = { uiHandler.post { updatePadMotion(); refreshControls() } }
steamController = SteamControllerBle(this).also { it.start() }
padMotion = com.droiddeck.launcher.input.PadMotion(this) {
@@ -420,6 +479,16 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
keyboard = KeyboardHost(this).also { root.addView(it) }
touchpad = TouchpadGestures(PointerGestures.slop(this), pointerListener)
threeFingerTap = com.droiddeck.launcher.input.ThreeFingerTap(PointerGestures.slop(this) * 2)
padMouse = PadMouse(padMouseListener)
bridge.setTouchRoute(object : PadBridge.TouchRoute {
override fun active(): Boolean = if (Looper.myLooper() == Looper.getMainLooper()) padMouseMode() else padMouseOn
override fun accept(state: PadState) {
if (Looper.myLooper() == Looper.getMainLooper()) padMouse.onTouchPad(state)
else PadState().also { it.copyFrom(state) }.let { copy -> uiHandler.post { padMouse.onTouchPad(copy) } }
}
})
scrollTouch = ScrollTouchGestures(PointerGestures.slop(this),
android.view.ViewConfiguration.get(this).scaledMinimumFlingVelocity.toFloat(), scrollListener)
val noCursor = android.view.PointerIcon.getSystemIcon(this, android.view.PointerIcon.TYPE_NULL)
root.pointerIcon = noCursor
surfaceView.pointerIcon = noCursor
@@ -450,6 +519,21 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
sessionOverlay = createSessionOverlay()
root.addView(sessionOverlay)
setContentView(root)
glassesScope.launch { Glasses.state.collect { glasses = it; onGlassesChanged() } }
glassesScope.launch { Glasses.settings.collect { glassesSettings = it; onGlassesChanged() } }
glassesScope.launch { com.droiddeck.launcher.handtracking.HandTracking.settings.collect { updateHandCamera() } }
glassesScope.launch {
while (true) {
if (SessionState.running && SessionState.mode == SessionService.MODE_STEAM &&
(SessionState.steamVr || steamVrStatus == SteamVr.Status.RUNNING)) {
val lost = kotlinx.coroutines.withContext(kotlinx.coroutines.Dispatchers.IO) {
com.droiddeck.launcher.gpu.SteamVrHealth.deviceLost()
}
if (lost) steamVrRecovery.detected()
}
kotlinx.coroutines.delay(2000)
}
}
pip = com.droiddeck.launcher.session.SessionPipController(this, ::pipBounds, ::showPipUi)
surfaceView.addOnLayoutChangeListener { _, _, _, _, _, _, _, _, _ -> pip.refresh() }
pip.refresh()
@@ -496,6 +580,15 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
CursorOverlay(cursorPos, cursorVisible, resources.displayMetrics.density,
cursorImage, cursorHotX, cursorHotY, cursorImageScale)
if (hud.text.isNotEmpty()) HudText(hud.text)
if (glassesOutputActive && !loading.visible) com.droiddeck.launcher.ui.GlassesPlayingHint(glasses.model)
if (GlassesRendering.usesGlasses(glasses, glassesOutputActive, steamVrStatus == SteamVr.Status.RUNNING) &&
headTracking && !drawerOpen && !pcKeyboardOpen && !loading.visible) com.droiddeck.launcher.ui.GlassesRecenterButton(
controllerSettings?.tint ?: android.graphics.Color.rgb(90, 160, 255), controllerSettings?.opacity ?: 100) { headTracker?.recenter() }
if (glasses.present && !drawerOpen && !pcKeyboardOpen && !loading.visible) com.droiddeck.launcher.ui.GlassesCornerButton(glassesSlot,
controllerSettings?.tint ?: android.graphics.Color.rgb(90, 160, 255), controllerSettings?.opacity ?: 100) {
drawerPage = com.droiddeck.launcher.ui.SessionDrawerPage.GLASSES
drawerOpen = true
}
if (loading.visible) LoadingOverlay(
loading.step, loading.percent, loading.elapsed, loading.hint, loading.ended,
topic = loading.topic, readable = loading.readable,
@@ -514,7 +607,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
androidx.compose.runtime.LaunchedEffect(drawerOpen) {
if (drawerOpen) {
syncLossless()
padBridge?.releaseAll()
releasePads()
androidx.compose.runtime.withFrameNanos { }
val requested = sessionOverlay.requestFocus()
Log.i(TAG, "drawer root focus requested=$requested focused=${sessionOverlay.hasFocus()}")
@@ -522,8 +615,12 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
releaseDrawerDirection()
}
}
LaunchedEffect(cardboard.enabled, steamVrSwitchedPicture, drawerOpen, drawerPage) {
while (cardboard.enabled || steamVrSwitchedPicture || (drawerOpen && drawerPage == SessionDrawerPage.VR)) {
val glassesVr = glasses.usable && glasses.canControl
val steamVrRequested = SessionState.steamVr
val pollSteamVr = SteamVr.shouldPoll(steamVrStatus, cardboard.enabled,
steamVrSwitchedPicture, glassesVr, steamVrRequested) || (drawerOpen && drawerPage == SessionDrawerPage.VR)
LaunchedEffect(pollSteamVr) {
while (pollSteamVr) {
val now = kotlinx.coroutines.withContext(kotlinx.coroutines.Dispatchers.IO) {
SteamVr.status(this@SessionActivity)
}
@@ -540,7 +637,10 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
achievementsInFront = found
if (found == null) achievementsOpen = false
}
androidx.compose.runtime.LaunchedEffect(pcKeyboardOpen) { if (pcKeyboardOpen) padBridge?.releaseAll() }
androidx.compose.runtime.LaunchedEffect(pcKeyboardOpen) { if (pcKeyboardOpen) releasePads() }
LaunchedEffect(pcKeyboardOpen, drawerOpen, achievementsOpen) {
if (pcKeyboardOpen || drawerOpen || achievementsOpen) padMouse.reset()
}
if (pcKeyboardOpen) com.droiddeck.launcher.ui.PcKeyboard(
sendKey = { code, down -> if (CompositorHost.isStarted) sendGuestKey(code, down) },
onAndroidKeyboard = { pcKeyboardOpen = false; keyboard?.toggle() },
@@ -559,6 +659,9 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
cardboardSensor = hasRotationSensor,
onCardboard = { c -> SessionPrefs.setCardboard(this@SessionActivity, c); applyCardboard(c) },
onCardboardRecenter = { headTracker?.recenter() },
glassesPresent = glasses.present,
glassesOutput = glassesOutputActive,
onGlassesRecenter = { headTracker?.recenter() },
steamVr = steamVrStatus,
immersiveSteamVr = com.droiddeck.launcher.session.QuestExperimental.isSupported(),
onSteamVr = { action -> steamVrAction(action) },
@@ -747,7 +850,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
))
LaunchedEffect(achievementsOpen) {
if (achievementsOpen) {
padBridge?.releaseAll()
releasePads()
withFrameNanos { }
sessionOverlay.requestFocus()
}
@@ -1108,7 +1211,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
com.droiddeck.launcher.frontend.Library.nameForProgram(program)?.let { getString(R.string.session_starting_named, it) }
?: getString(R.string.session_starting_program)
}
else -> getString(R.string.load_starting_steam)
else -> if (com.droiddeck.launcher.session.SessionPrefs.activeSteamClientArch(this) == "x86_64") getString(R.string.load_starting_steam_x64)
else getString(R.string.load_starting_steam)
}
private fun shareCurrentSessionLogs() {
@@ -1148,6 +1252,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
val nextTouchMode = SessionPrefs.touchMode(this)
if (touchMode != nextTouchMode) {
if (::touchpad.isInitialized) touchpad.cancel()
if (::scrollTouch.isInitialized) scrollTouch.cancel()
if (CompositorHost.isStarted) {
WaylandCompositor.nativeSendTouch(3, -1, 0, 0)
listOf(PointerGestures.BTN_LEFT, PointerGestures.BTN_RIGHT, PointerGestures.BTN_MIDDLE).forEach {
@@ -1224,7 +1329,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
loading.percent = -1
loading.say(getString(R.string.session_starting_session), LoadingState.Topic.SESSION, readable = false)
if (surfaceView.holder.surface?.isValid == true) surfaceCreated(surfaceView.holder)
outputHolder().let { if (it.surface?.isValid == true) surfaceCreated(it) }
}
}, "runtime-install").start()
}
@@ -1288,7 +1393,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
override fun surfaceCreated(holder: SurfaceHolder) {
if (installingRuntime || SessionState.stopRequested) return
if (holder !== outputHolder()) return
if (installingRuntime || SessionState.stopRequested || ended) return
if (!SessionState.running) {
SessionEvents.transition(SessionPhase.STARTING_COMPOSITOR, "compositor.starting")
}
@@ -1304,7 +1410,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
val live = if (SessionState.followScreen && !isInPictureInPictureMode)
com.droiddeck.launcher.session.SessionDisplay.followSize(holder.surfaceFrame.width(), holder.surfaceFrame.height())
else null
val size = live ?: if (SessionState.running) previous else outputSize()
val size = live ?: if (SessionState.running) previous else glassesOutputSize() ?: outputSize()
SessionState.outputSize = size
if (!SessionState.running) SessionState.refreshHz = refreshHz()
val panel = panelBounds()
@@ -1317,7 +1423,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
WaylandCompositor.nativeSetUpscaler(SessionPrefs.upscaler(this))
WaylandCompositor.nativeSetUpscaleSharpness(SessionPrefs.upscaleSharpness(this))
SessionPrefs.screenEffects(this).push(SessionPrefs.upscaler(this))
panelSizeMm().let { (w, h) -> SessionPrefs.cardboard(this).push(w, h) }
activeCardboard = effectiveCardboard(SessionPrefs.cardboard(this))
pushCardboard(activeCardboard)
WaylandCompositor.nativeSetCardboardPoseFile(File(filesDir, ".wayland-rt/droiddeck-headpose").path)
if (!SessionState.running) {
val waylandLog = File(SessionPaths.beginOrCurrent(this), "wayland.log")
@@ -1330,7 +1437,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
if (!CompositorHost.isStarted) {
val mode = SessionPrefs.prefMode(intent.getStringExtra(SessionService.EXTRA_MODE) ?: SessionService.MODE_STEAM)
val probe = HdrSupport.probe(this)
val probe = HdrSupport.probe(this, glassesScreen?.display)
WaylandCompositor.nativeSetHdrDisplay(
probe.displayId, probe.name, probe.formats, probe.hdr10,
probe.maxLuminance, probe.maxAverageLuminance, probe.minLuminance,
@@ -1354,8 +1461,9 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
CompositorHost.startOrAttach(
holder.surface, runtimeDir.path,
driverId?.let { turnip.driverPath(it) }, driverId?.let { turnip.libraryName(it) },
applicationInfo.nativeLibraryDir, size.first, size.second, refreshHz(), SessionState.fpsLimit,
applicationInfo.nativeLibraryDir, size.first, size.second, refreshHz(), outputFpsLimit(),
)
onOutputAttached(holder)
if (SessionState.running && size != previous) {
Log.i(TAG, "follow screen: ${previous.first}x${previous.second} -> ${size.first}x${size.second} on a new surface")
WaylandCompositor.nativeResizeOutput(size.first, size.second)
@@ -1379,6 +1487,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
private var surfaceH = 0
override fun surfaceChanged(holder: SurfaceHolder, format: Int, width: Int, height: Int) {
if (holder !== outputHolder()) return
val resized = surfaceW != 0 && (width != surfaceW || height != surfaceH)
surfaceW = width
surfaceH = height
@@ -1422,8 +1531,12 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
override fun surfaceDestroyed(holder: SurfaceHolder) {
// Hiding the phone surface after a handoff delivers a late destruction
// callback. It must not reset the new output's size or attachment.
if (holder !== outputHolder()) return
surfaceW = 0
surfaceH = 0
glassesOutputActive = false
CompositorHost.detach(holder.surface)
}
@@ -1448,14 +1561,32 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
private fun refreshHz(): Float {
glassesScreen?.hz?.takeIf { it > 1f }?.let { return it }
val display = if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay
val wanted = window.attributes.preferredDisplayModeId
val hz = display?.supportedModes?.firstOrNull { wanted != 0 && it.modeId == wanted }?.refreshRate
?: display?.refreshRate ?: 60f
val capped = minOf(hz, PerfMode.peakRefreshCap(this))
val mirrorCap = mirrorRefreshLimit().takeIf { it > 0 }?.toFloat() ?: Float.POSITIVE_INFINITY
val capped = minOf(hz, PerfMode.peakRefreshCap(this), mirrorCap)
return if (capped > 1f) capped else 60f
}
private fun mirrorRefreshLimit(): Int =
GlassesRendering.mirrorRefreshLimit(glassesSettings, glasses, glassesScreen != null)
private fun outputFpsLimit(): Int = GlassesRendering.fpsLimit(SessionState.fpsLimit, mirrorRefreshLimit())
private fun applyOutputPacing() {
Log.i(TAG, "perf: " + PerfMode.apply(this, outputFpsLimit(), mirrorRefreshLimit()))
if (CompositorHost.isStarted) {
val hz = refreshHz()
val changed = kotlin.math.abs(SessionState.refreshHz - hz) > 0.1f
CompositorHost.updatePacing(hz, outputFpsLimit())
pushCardboard(activeCardboard)
if (changed && ::surfaceView.isInitialized) applyFrameGen()
}
}
private fun applyFrameGen() {
val hz = refreshHz()
Thread({
@@ -1514,11 +1645,14 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
private fun onSessionEnded(status: Int) {
ended = true
if (isInPictureInPictureMode) {
runOnUiThread { finish() }
return
}
closeSecondScreen(reset = true)
updateGlassesScreen()
updateHeadTracker()
if (status == 0) {
runOnUiThread {
if (isFinishing || isDestroyed) return@runOnUiThread
@@ -1560,9 +1694,14 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
focusEndedScreen()
}
private fun releasePads() {
padBridge?.releaseAll()
joyConVr?.releaseButtons()
}
private fun focusEndedScreen() {
drawerOpen = false
padBridge?.releaseAll()
releasePads()
uiHandler.post { sessionOverlay.requestFocus() }
}
@@ -1610,6 +1749,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
return super.dispatchKeyEvent(event)
}
if (pipUi) return true
val pad = com.droiddeck.launcher.input.JoyCon.normalize(event)
if (pad !== event) return dispatchKeyEvent(pad)
val fromController = PadBridge.isControllerKeyEvent(event)
if (fromController && event.action == KeyEvent.ACTION_DOWN) {
if (drawerOpen && !drawerControllerActive) sessionOverlay.requestFocus()
@@ -1632,7 +1773,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
if (drawerOpen) {
if (fromController && (event.keyCode == KeyEvent.KEYCODE_BUTTON_L1 || event.keyCode == KeyEvent.KEYCODE_BUTTON_R1)) {
if (event.action == KeyEvent.ACTION_DOWN && event.repeatCount == 0) {
drawerPage = sessionDrawerPages(SessionState.mode == SessionService.MODE_STEAM && secondScreenDisplays.isNotEmpty())
drawerPage = sessionDrawerPages(SessionState.mode == SessionService.MODE_STEAM && secondScreenDisplays.isNotEmpty(), glasses.present)
.step(drawerPage, if (event.keyCode == KeyEvent.KEYCODE_BUTTON_R1) 1 else -1)
releaseDrawerDirection()
}
@@ -1661,6 +1802,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
return true
}
if ((pcKeyboardOpen || achievementsOpen) && fromController) return super.dispatchKeyEvent(event)
if (joyConVr?.onKeyEvent(event) == true) return true
if (fromController && event.keyCode != KeyEvent.KEYCODE_BACK && padMouseMode() && padMouse.onKey(event)) return true
if (event.keyCode != KeyEvent.KEYCODE_BACK && padBridge?.onKeyEvent(event) == true) return true
if (CompositorHost.isStarted && event.keyCode != KeyEvent.KEYCODE_BACK && !fromController) {
val down = event.action == KeyEvent.ACTION_DOWN
@@ -1704,6 +1847,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
if (drawerDirectionKey != KeyEvent.KEYCODE_UNKNOWN) releaseDrawerDirection()
if ((pcKeyboardOpen || achievementsOpen) && PadBridge.isControllerMotionEvent(event)) return super.dispatchGenericMotionEvent(event)
if (joyConVr?.onMotionEvent(event) == true) return true
if (PadBridge.isControllerMotionEvent(event) && padMouseMode() && padMouse.onMotion(event)) return true
if (padBridge?.onMotionEvent(event) == true) return true
if (event.isFromSource(android.view.InputDevice.SOURCE_MOUSE) && !drawerOpen && onMouse(event)) return true
return super.dispatchGenericMotionEvent(event)
@@ -1806,6 +1951,67 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
}
private val scrollListener = object : ScrollTouchGestures.Listener {
override fun onMove(x: Float, y: Float) {
val rect = drawnRect() ?: return
val cx = x.coerceIn(rect.left, rect.right)
val cy = y.coerceIn(rect.top, rect.bottom)
WaylandCompositor.nativeSendPointer(1, InputSpace.x(cx, surfaceView.width), InputSpace.y(cy, surfaceView.height))
cursorVisible = false
}
override fun onButton(button: Int, pressed: Boolean) {
WaylandCompositor.nativeSendSceneInput(3, button, if (pressed) 1 else 0)
}
override fun onScroll(dx: Float, dy: Float) {
val rect = drawnRect() ?: return
val out = SessionState.outputSize
scrollRestX += dx * out.first / rect.width()
scrollRestY += dy * out.second / rect.height()
val h = scrollRestX.toInt()
val v = scrollRestY.toInt()
if (h == 0 && v == 0) return
scrollRestX -= h
scrollRestY -= v
WaylandCompositor.nativeSendSceneInput(11, ScrollTouchGestures.wheelUnits(v, out.second), ScrollTouchGestures.wheelUnits(h, out.first))
}
override fun onLongPress() = pointerListener.onLongPress()
}
private val padMouseListener = object : PadMouse.Listener {
override fun active(): Boolean = padMouseMode()
override fun onScrollStart() {
scrollRestX = 0f
scrollRestY = 0f
val rect = drawnRect() ?: return
val x = rect.left + rect.width() * PAD_SCROLL_POINTER_X
WaylandCompositor.nativeSendPointer(1, InputSpace.x(x, surfaceView.width), InputSpace.y(rect.centerY(), surfaceView.height))
cursorVisible = false
}
override fun onScroll(dx: Float, dy: Float) = scrollListener.onScroll(dx, dy)
override fun onKey(evdev: Int, pressed: Boolean) = sendGuestKey(evdev, pressed)
override fun onKeyboard() {
uiHandler.post { togglePcKeyboard() }
}
override fun onNavigate() {
cursorVisible = false
}
override fun scrollSpeed(): Float = (drawnRect()?.height() ?: 0f) * PAD_SCROLL_HEIGHTS_PER_S
}
private fun padMouseMode(): Boolean {
val now = android.os.SystemClock.uptimeMillis()
if (now - padMouseCheckedAt < PAD_MOUSE_CHECK_MS) return padMouseOn
padMouseCheckedAt = now
val on = !pointerLocked && EpicClient.launcherFocused(this) && !EpicClient.playing(this)
if (padMouseOn && !on) padMouse.reset()
if (!padMouseOn && on) padBridge?.releaseAll()
padMouseOn = on
return on
}
private fun scrollTouchMode(): Boolean =
(routedTouchMode == SessionPrefs.TOUCH_AUTO || routedTouchMode == SessionPrefs.TOUCH_DIRECT) && padMouseMode()
private fun moveLockedPointer(x: Float, y: Float) {
val rect = drawnRect() ?: return
val scale = SessionState.outputSize.first / rect.width()
@@ -1868,6 +2074,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
if (!::displayManager.isInitialized) return
val primaryId = display?.displayId ?: windowManager.defaultDisplay.displayId
val candidates = SecondScreenDisplays.available(displayManager, primaryId)
.filter { glassesScreen == null || it.id != glasses.displayId }
val oldIds = secondScreenDisplays.map { it.id }.toSet()
secondScreenDisplays = candidates
val newIds = candidates.map { it.id }.toSet()
@@ -2032,6 +2239,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
if (next == routedTouchMode) return
if (::touchpad.isInitialized) touchpad.cancel()
if (::scrollTouch.isInitialized) scrollTouch.cancel()
if (CompositorHost.isStarted) WaylandCompositor.nativeSendTouch(3, -1, 0, 0)
if (next == SessionPrefs.TOUCH_OFF) cursorVisible = false
routedTouchMode = next
@@ -2065,6 +2273,12 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
override fun onTouchEvent(event: MotionEvent): Boolean {
if (event.isFromSource(android.view.InputDevice.SOURCE_MOUSE)) return onMouse(event)
if (event.actionMasked == MotionEvent.ACTION_DOWN) {
scrollTouchGesture = scrollTouchMode()
scrollRestX = 0f
scrollRestY = 0f
}
if (scrollTouchGesture) return scrollTouch.onTouch(event)
if (routedTouchMode == SessionPrefs.TOUCH_OFF) return true
if (SessionState.mode == SessionService.MODE_STEAM && SessionState.steamUi != "desktop") {
val wasActive = threeFingerTap.active
@@ -2219,6 +2433,17 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
private fun updateOnScreenControls() {
updateOnScreenPad()
val c = onScreenControls
glassesSlot = when {
c == null || c.visibility != View.VISIBLE -> 0
c.isButtonsOnly -> -1
controllerSettings?.keyboardButton != false -> 2
else -> 1
}
}
private fun updateOnScreenPad() {
val forced = File(Environment.getExternalStorageDirectory(), "Download/droiddeck-osc")
.takeIf { it.isFile }
?.let { FileUtils.readString(it)?.trim()?.lowercase() }
@@ -2301,8 +2526,9 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
pendingBackAction = null
releaseDrawerDirection()
onScreenControls?.releaseAll()
padBridge?.releaseAll()
releasePads()
touchpad.cancel()
scrollTouch.cancel()
guestKeysDown.toList().forEach { sendGuestKey(it, false) }
keyboard?.hide()
sessionClipboard.stop()
@@ -2337,6 +2563,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
if (::pip.isInitialized) pip.changed(inPip)
if (!inPip && surfaceW > 0) followWindow(surfaceW, surfaceH)
applyUnfoldedControls()
updateGlassesScreen()
}
override fun startActivityForResult(intent: Intent, requestCode: Int, options: Bundle?) {
@@ -2354,15 +2581,22 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
displayManager.registerDisplayListener(displayListener, Handler(Looper.getMainLooper()))
refreshSecondScreenDisplays()
if (secondScreenMode != SecondScreenMode.NONE && secondScreenPresentation == null) showSecondScreen(secondScreenMode)
startedScreens++
SessionService.setActivityVisible(this, true)
Glasses.rescan()
updateGlassesScreen()
updateHandCamera()
}
override fun onStop() {
if (runtimeRemovalBlocked) { super.onStop(); return }
started = false
updateHandCamera()
displayManager.unregisterDisplayListener(displayListener)
closeSecondScreen(reset = false)
SessionService.setActivityVisible(this, false)
startedScreens = (startedScreens - 1).coerceAtLeast(0)
updateGlassesScreen()
if (startedScreens == 0) SessionService.setActivityVisible(this, false)
super.onStop()
}
@@ -2379,6 +2613,10 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
if (this.intent.getBooleanExtra(EXTRA_RETURN_HOME, false)) intent.putExtra(EXTRA_RETURN_HOME, true)
setIntent(intent)
if (intent.getBooleanExtra(EXTRA_STEAMVR, false) && SessionState.mode == SessionService.MODE_STEAM) {
SessionState.steamVr = true
steamVrAutoTried = false
}
if (intent.action == SessionService.ACTION_HOME_GUIDE) {
handleHomeGuideIntent(intent)
} else if (intent.action == SessionService.ACTION_RESUME) {
@@ -2400,6 +2638,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
(getSystemService(INPUT_SERVICE) as? InputManager)
?.registerInputDeviceListener(deviceListener, Handler(Looper.getMainLooper()))
if (SessionState.guestPid > 1) padBridge?.syncConnectedControllers()
joyConVr?.refresh()
onScreenControls?.reload()
controllerSettings = ControllerPrefs.read(this)
resumed = true
@@ -2422,6 +2661,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
(getSystemService(INPUT_SERVICE) as? InputManager)?.unregisterInputDeviceListener(deviceListener)
releaseDrawerDirection()
onScreenControls?.releaseAll()
if (::scrollTouch.isInitialized) scrollTouch.cancel()
if (::padMouse.isInitialized) padMouse.reset()
resumed = false
updatePointerCapture()
updatePadMotion()
@@ -2436,13 +2677,32 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
startActivity(android.content.Intent(this, QuestXrActivity::class.java))
return
}
if (action == "start") {
SessionState.steamVr = true
steamVrAutoTried = true
}
if (action == "stop") SessionState.steamVr = false
if (glasses.present) {
// Optional stereo starts only after RUNNING is observed. READY can
// arrive repeatedly while the launch worker is still starting VR.
if (action == "install") steamVrStatus = SteamVr.Status.INSTALLING
steamVrWorker.execute {
val ok = SteamVr.run(this, action)
val now = SteamVr.status(this)
uiHandler.post {
onSteamVrStatus(now)
if (!ok) Toast.makeText(this, getString(R.string.cardboard_steamvr_failed), Toast.LENGTH_LONG).show()
}
}
return
}
if (action == "start") {
switchPictureForSteamVr()
steamVrSeenRunning = false
}
if (action == "stop") restoreSteamVrPicture()
if (action == "install") steamVrStatus = SteamVr.Status.INSTALLING
Thread({
steamVrWorker.execute {
val ok = SteamVr.run(this, action)
val now = SteamVr.status(this)
uiHandler.post {
@@ -2450,7 +2710,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
onSteamVrStatus(now)
if (!ok) Toast.makeText(this, getString(R.string.cardboard_steamvr_failed), Toast.LENGTH_LONG).show()
}
}, "steamvr-$action").start()
}
}
private fun switchPictureForSteamVr() {
@@ -2463,8 +2723,41 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
steamVrSwitchedPicture = SessionPrefs.cardboardSourceBeforeSteamVr(this) >= 0
}
private fun updateHandCamera() {
val joyCons = joyConVr?.holding == true
val bare = !joyCons && !controllerConnected()
val want = started && steamVrStatus == SteamVr.Status.RUNNING && (joyCons || bare) &&
com.droiddeck.launcher.handtracking.HandTracking.wanted(this)
handCamera?.bareHands = bare
if (want == (handCamera != null)) return
if (want) {
handCamera = com.droiddeck.launcher.handtracking.HandCamera(this,
File(filesDir, ".wayland-rt/${com.droiddeck.launcher.handtracking.HandCameraShm.FILE}"), preview = false)
.also {
it.bareHands = bare
it.start(com.droiddeck.launcher.handtracking.HandTracking.settings.value.side)
}
} else {
handCamera?.stop()
handCamera = null
}
}
private fun controllerConnected(): Boolean = InputDevice.getDeviceIds().any { id ->
joyConVr?.owns(id) != true && com.droiddeck.launcher.input.PadBridge.isFromController(InputDevice.getDevice(id))
}
private fun onSteamVrStatus(now: SteamVr.Status) {
val was = steamVrStatus
steamVrStatus = now
joyConVr?.setActive(now == SteamVr.Status.RUNNING && ControllerPrefs.joyConVr(this))
updateHandCamera()
autoStartSteamVr(now)
if ((was == SteamVr.Status.RUNNING) != (now == SteamVr.Status.RUNNING)) applyCardboard(cardboard)
if (glasses.present) onGlassesSteamVr(now)
if (GlassesRendering.usesGlasses(glasses, glassesScreen != null, now == SteamVr.Status.RUNNING)) {
return
}
if (now == SteamVr.Status.RUNNING && cardboard.enabled && !steamVrSwitchedPicture &&
cardboard.source != Cardboard.SOURCE_SBS) {
switchPictureForSteamVr()
@@ -2487,23 +2780,172 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
private fun applyCardboard(c: Cardboard) {
val was = cardboard
val was = activeCardboard
cardboard = c
if (CompositorHost.isStarted) panelSizeMm().let { (w, h) -> c.push(w, h) }
if (c.enabled && !was.enabled) headTracker?.recenter()
val e = effectiveCardboard(c)
activeCardboard = e
if (CompositorHost.isStarted) pushCardboard(e)
if (e.enabled && !was.enabled) headTracker?.recenter()
updateHeadTracker()
}
private fun pushCardboard(c: Cardboard) {
val fov = when {
c.viewer != Cardboard.Viewer.VITURE.id -> 0f
c.source == Cardboard.SOURCE_SBS -> glassesSettings.vrFovDeg.toFloat()
else -> glasses.verticalFovDeg
}
panelSizeMm().let { (w, h) -> c.push(w, h, fov, refreshHz()) }
}
private fun effectiveCardboard(c: Cardboard): Cardboard = GlassesRendering.cardboard(
base = c,
settings = glassesSettings,
state = glasses,
outputActive = glassesScreen != null,
steamVrRunning = steamVrStatus == SteamVr.Status.RUNNING,
)
private fun updateHeadTracker() {
val want = resumed && cardboard.enabled && cardboard.headTracking
val onGlasses = GlassesRendering.usesGlasses(glasses, glassesScreen != null,
steamVrStatus == SteamVr.Status.RUNNING)
val c = activeCardboard
val want = !ended && (resumed || (glassesScreen != null && started)) && c.enabled && c.headTracking
val current = headTracker
if (current != null && (!want || current.usesGlasses != onGlasses)) {
current.stop()
headTracker = null
}
if (want) {
val t = headTracker ?: HeadTracker(this) {
val t = headTracker ?: HeadTracker(this, glasses = onGlasses) {
@Suppress("DEPRECATION")
(if (Build.VERSION.SDK_INT >= 30) display else windowManager.defaultDisplay)?.rotation ?: android.view.Surface.ROTATION_0
}.also { headTracker = it }
t.start()
} else {
headTracker?.stop()
}
headTracking = headTracker != null
}
private fun outputHolder(): SurfaceHolder = glassesScreen?.holder ?: surfaceView.holder
private fun onOutputAttached(holder: SurfaceHolder) {
if (holder !== outputHolder()) return
glassesOutputActive = glassesScreen?.holder === holder
// Keep the phone's layout for touch/controller coordinates, but remove
// its SurfaceView so it cannot retain the last game frame under the hint.
surfaceView.visibility = if (glassesOutputActive) View.INVISIBLE else View.VISIBLE
Log.i(TAG, "compositor output attached: " +
if (glassesOutputActive) "glasses display ${glassesScreen?.display?.displayId}" else "phone")
}
private fun restorePhoneOutput(screen: GlassesScreen) {
if (glassesScreen !== screen) return
glassesScreen = null
glassesOutputActive = false
surfaceW = 0
surfaceH = 0
CompositorHost.detach(screen.holder.surface)
surfaceView.visibility = View.VISIBLE
applyOutputPacing()
if (surfaceView.holder.surface?.isValid == true) surfaceCreated(surfaceView.holder)
}
private fun glassesOutputSize(): Pair<Int, Int>? {
if (glassesScreen == null || glasses.width <= 0 || glasses.height <= 0) return null
val panel = (if (glasses.sbs) glasses.width / 2 else glasses.width) to glasses.height
val mode = SessionPrefs.prefMode(intent.getStringExtra(SessionService.EXTRA_MODE) ?: SessionService.MODE_STEAM)
val display = com.droiddeck.launcher.session.SessionDisplay
val choice = SessionPrefs.resolutionChoice(this, mode, panel)
val size = display.resolveChoice(panel, choice)
val fixed = choice != display.MATCH_SCREEN && choice != display.FOLLOW_SCREEN && display.presetHeight(choice) == null
return if (fixed) display.resolve(panel, minOf(size.first, size.second), SessionPrefs.SHAPE_AUTO) else size
}
private fun onGlassesChanged() {
val g = glasses
val s = glassesSettings
val key = listOf(g.present, g.usable, g.displayId, g.sbs, g.sbs90, g.switching3d, g.canControl, g.width, g.height, g.hz, s)
if (key == glassesKey) return
glassesKey = key
updateGlassesScreen()
applyCardboard(cardboard)
if (!g.switching3d) {
if (glassesPresent && !g.present) onGlassesUnplugged()
glassesPresent = g.present
}
}
private fun onGlassesUnplugged() {
val hadVr = steamVrAutoTried
steamVrAutoTried = false
glasses3dForVr = false
if (hadVr && !cardboard.enabled) steamVrAction("stop")
}
private fun autoStartSteamVr(now: SteamVr.Status) {
if (now == SteamVr.Status.RUNNING) steamVrAutoTried = true
val action = SteamVr.autoAction(now, SessionState.steamVr, loading.visible,
SessionState.mode == SessionService.MODE_STEAM, steamVrAutoTried, steamVrInstallTried) ?: return
if (action == "start") steamVrAutoTried = true else steamVrInstallTried = true
Log.i(TAG, "requested SteamVR $action (glasses=${glasses.model}, output=$glassesOutputActive, control=${glasses.canControl})")
steamVrAction(action)
}
private fun updateGlassesScreen() {
if (!::surfaceView.isInitialized) return
val g = glasses
val want = started && !ended && !isInPictureInPictureMode && !runtimeRemovalBlocked && g.usable && glassesSettings.gameOnGlasses
val target = if (want) Glasses.glassesDisplay()?.takeIf { it.isValid } else null
val current = glassesScreen
if (current != null && current.display.displayId == target?.displayId) {
current.prefer(Glasses.outputHz())
applyOutputPacing()
return
}
if (current != null) {
restorePhoneOutput(current)
runCatching { current.dismiss() }
}
if (target != null) {
val screen = GlassesScreen(this, target, Glasses.outputHz(), this)
// show() can create the surface synchronously. Select its holder
// first, otherwise surfaceCreated() rejects the glasses callback.
glassesScreen = screen
applyOutputPacing()
surfaceW = 0
surfaceH = 0
screen.setOnDismissListener {
if (glassesScreen === screen) {
restorePhoneOutput(screen)
refreshSecondScreenDisplays()
applyCardboard(cardboard)
}
}
try {
screen.show()
} catch (e: Exception) {
restorePhoneOutput(screen)
runCatching { screen.dismiss() }
Log.w(TAG, "could not show the game on ${target.name}", e)
return
}
Log.i(TAG, "requested game output on ${g.model} (display ${target.displayId})")
}
applyOutputPacing()
refreshSecondScreenDisplays()
applyCardboard(cardboard)
}
private fun onGlassesSteamVr(now: SteamVr.Status) {
val g = glasses
if (now == SteamVr.Status.RUNNING) {
if (glassesSettings.auto3d && g.can3d && !g.sbs && !glasses3dForVr) {
glasses3dForVr = true
Glasses.set3dForVr(true)
}
} else if (glasses3dForVr && now != SteamVr.Status.UNAVAILABLE) {
glasses3dForVr = false
Glasses.set3dForVr(false)
}
}
@@ -2535,6 +2977,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
override fun onWindowFocusChanged(hasFocus: Boolean) {
super.onWindowFocusChanged(hasFocus)
if (!hasFocus && ::padMouse.isInitialized) padMouse.reset()
if (runtimeRemovalBlocked) return
if (hasFocus && !pipUi) {
goFullscreen()
@@ -2575,6 +3018,7 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
}
override fun onDestroy() {
steamVrRecovery.close()
uiHandler.removeCallbacks(steamTouchRetry)
if (runtimeRemovalBlocked) { super.onDestroy(); return }
watching = false
@@ -2583,7 +3027,14 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
pendingBackAction = null
if (::pip.isInitialized) pip.destroyed()
closeSecondScreen(reset = false)
glassesScope.cancel()
glassesScreen?.let { screen -> glassesScreen = null; runCatching { screen.dismiss() } }
headTracker?.stop()
if (::hud.isInitialized) hud.stop()
joyConVr?.close()
joyConVr = null
handCamera?.stop()
handCamera = null
padBridge?.stop()
steamController?.stop()
if (SessionState.padBridge === padBridge) SessionState.padBridge = null
@@ -2637,11 +3088,16 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
companion object {
const val EXTRA_RETURN_HOME = "returnHome"
const val EXTRA_STEAMVR = "steamVr"
private const val LEAVE_TIMEOUT_MS = 1_500L
private const val TAG = "SessionActivity"
private var startedScreens = 0
private const val FEX_MISSING = "droiddeck-fex: x86 Linux programs need"
private const val UNBUFFERED_SOURCES = InputDevice.SOURCE_CLASS_JOYSTICK or InputDevice.SOURCE_CLASS_TRACKBALL or InputDevice.SOURCE_CLASS_POSITION
private const val CURSOR_PAD_HOLD_MS = 1200L
private const val PAD_MOUSE_CHECK_MS = 500L
private const val PAD_SCROLL_POINTER_X = 0.985f
private const val PAD_SCROLL_HEIGHTS_PER_S = 3f
private const val DRAWER_HAT_THRESHOLD = 0.5f
private const val DRAWER_STICK_ENTER_THRESHOLD = 0.55f
private const val DRAWER_STICK_EXIT_THRESHOLD = 0.35f
@@ -2650,6 +3106,8 @@ class SessionActivity : ComponentActivity(), SurfaceHolder.Callback {
private const val DRAWER_SLOW_REPEAT_FLOOR_MS = 240L
private const val DRAWER_FAST_REPEAT_INTERVAL_MS = 90L
private const val DRAWER_REPEAT_ACCELERATION_MS = 1_400L
private val steamVrWorker = java.util.concurrent.ThreadPoolExecutor(0, 1, 30, java.util.concurrent.TimeUnit.SECONDS,
java.util.concurrent.LinkedBlockingQueue()) { Thread(it, "steamvr") }
private const val SCALE_FIT = 1
private const val ALIGN_CENTER = 0
private const val ALIGN_PICTURE_TOP = 3
@@ -92,11 +92,75 @@ class AgentBridgeProvider : ContentProvider() {
AgentEnv.override(context, request.optString("name")).put("ok", true).put("command", "override")
}
"prefs" -> prefs(context, request)
"glasses" -> glasses(context, request)
"access" -> JSONObject().put("ok", true).put("commands", AgentAccess.commandsAllowed(context))
.put("debugBuild", BuildConfig.DEBUG).put("inbox", AgentEnv.inbox(context).absolutePath)
else -> error("UNKNOWN_COMMAND", "Unknown agent command '$method'")
}
private fun glasses(context: Context, request: JSONObject): JSONObject {
val g = com.droiddeck.launcher.glasses.Glasses
when (request.optString("op", "status")) {
"status" -> {}
"allow" -> g.requestControl()
"set" -> {
val key = request.optString("key")
val value = request.optString("value")
fun on() = value == "1" || value.equals("true", true) || value.equals("on", true)
val number = value.toIntOrNull()
val numeric = setOf("refreshHz", "sbsHz", "brightness", "volume", "picture3d", "screenWidthDm", "screenDistanceDm", "vrFovDeg", "sbsSupport")
if (key in numeric && number == null) return error("INVALID_VALUE", "'$key' takes a number")
when (key) {
"refreshHz" -> g.setRefreshHz(number!!)
"sbsHz" -> g.setSbsHz(number!!)
"brightness" -> g.setBrightness(number!!)
"volume" -> g.setVolume(number!!)
"sunblock" -> g.setSunblock(on())
"3d" -> g.set3d(on())
"gameOnGlasses" -> g.setGameOnGlasses(on())
"headTracking" -> g.setHeadTracking(on())
"auto3d" -> g.setAuto3d(on())
"picture3d" -> g.setPicture3d(number!!)
"screenWidthDm" -> g.setScreenWidthDm(number!!)
"screenDistanceDm" -> g.setScreenDistanceDm(number!!)
"vrFovDeg" -> g.setVrFovDeg(number!!)
"sbsSupport" -> g.setSbsSupport(number!!)
else -> return error("INVALID_KEY", "Unknown glasses setting '$key'")
}
}
"simulate" -> {
fun angle(key: String) = if (request.has(key)) request.optDouble(key).takeIf { it.isFinite() }?.coerceIn(-90.0, 90.0) else null
if (!g.simulate(request.optInt("seconds", 20).coerceIn(1, 600), angle("yaw"), angle("pitch"))) return error("BUSY", "Already simulating")
}
"stop-simulate" -> g.stopSimulation()
"calibrate" -> g.startCalibration()
"cancel-calibration" -> g.cancelCalibration()
"reset-calibration" -> g.resetCalibration()
"query" -> { g.query(); Thread.sleep(600) }
"mode" -> {
val mode = hex(request.optString("mode"), 0xFF) ?: return error("INVALID_VALUE", "mode must be one hex byte, e.g. 0x31")
g.sendMode(mode)
Thread.sleep(600)
}
"imu" -> return g.sampleImu(request.optLong("ms", 2000).coerceIn(200, 20_000)).put("ok", true).put("command", "glasses")
"raw" -> {
AgentAccess.requireCommands(context)
val msgId = hex(request.optString("msgId"), 0xFFFF) ?: return error("INVALID_VALUE", "msgId must be a hex number up to 0xFFFF")
val digits = request.optString("payload").removePrefix("0x")
if (digits.length % 2 != 0 || digits.any { Character.digit(it, 16) < 0 }) return error("INVALID_VALUE", "payload must be whole hex bytes")
val payload = digits.chunked(2).map { it.toInt(16).toByte() }.toByteArray()
if (!g.sendRaw(msgId, payload)) return error("NOT_CONNECTED", "The glasses' control link is not open")
Thread.sleep(300)
}
else -> return error("INVALID_OP", "Unknown glasses op")
}
if (request.optString("op") in setOf("set", "allow")) Thread.sleep(400)
return g.status().put("ok", true).put("command", "glasses").put("frames", g.recentFrames())
}
private fun hex(text: String, max: Int): Int? =
text.removePrefix("0x").removePrefix("0X").takeIf { it.isNotEmpty() }?.toIntOrNull(16)?.takeIf { it in 0..max }
private fun state(context: Context): JSONObject {
val runtimeVersion = LinuxRuntimeInstaller.installedVersion(context)
val dir = SessionState.logDirectory ?: SessionState.logFile?.parentFile ?: SessionPaths.sessionFolders(context).lastOrNull()
@@ -109,6 +173,7 @@ class AgentBridgeProvider : ContentProvider() {
.put("steamUi", SessionState.steamUi ?: JSONObject.NULL)
.put("steamUrl", SessionState.steamUrl ?: JSONObject.NULL)
.put("suspended", SessionState.suspended)
.put("steamVr", SessionState.steamVr)
.put("pip", SessionState.pipActive)
.put("firstFrame", SessionState.firstFrameSeen)
.put("output", JSONArray().put(SessionState.outputSize.first).put(SessionState.outputSize.second))
@@ -136,8 +136,12 @@ object AgentGuest {
return null
}
fun call(context: Context, request: JSONObject, timeoutMs: Long): JSONObject {
if (request.optString("kind") in setOf("exec", "cdp")) AgentAccess.requireCommands(context)
fun steamEval(context: Context, expression: String, timeoutMs: Long): JSONObject =
call(context, JSONObject().put("kind", "cdp").put("expression", expression).put("timeout", timeoutMs / 1000.0),
timeoutMs + 2_000, trusted = true)
fun call(context: Context, request: JSONObject, timeoutMs: Long, trusted: Boolean = false): JSONObject {
if (!trusted && request.optString("kind") in setOf("exec", "cdp")) AgentAccess.requireCommands(context)
if (!SessionState.running) throw AgentException("NO_ACTIVE_SESSION", "There is no running session")
hello(context) ?: throw AgentException("AGENT_UNAVAILABLE", "The guest agent is not running in this session (yet)")
val dir = dir(context)
@@ -97,6 +97,7 @@ class AgentStartActivity : Activity() {
request.optString("steamUrl").takeIf { it.startsWith("steam://") }?.let {
putExtra(SessionService.EXTRA_STEAM_URL, it)
}
if (request.optBoolean("steamVr")) putExtra(SessionActivity.EXTRA_STEAMVR, true)
}
}
startActivity(sessionIntent)
@@ -2,10 +2,15 @@ package com.droiddeck.launcher.core
import android.app.ActivityManager
import android.content.Context
import android.hardware.display.DisplayManager
import android.os.Build
import android.os.Environment
import android.os.PerformanceHintManager
import android.os.PowerManager
import android.os.StatFs
import android.provider.Settings
import android.util.Log
import android.view.Display
import com.droiddeck.launcher.gpu.LinuxVulkanDriver
import com.droiddeck.launcher.gpu.LinuxVulkanDriverManager
import com.droiddeck.launcher.gpu.TurnipDriver
@@ -54,6 +59,29 @@ object DeviceReport {
else -> "REASON_$reason"
}
internal fun thermalStatusName(status: Int): String = when (status) {
0 -> "NONE"
1 -> "LIGHT"
2 -> "MODERATE"
3 -> "SEVERE"
4 -> "CRITICAL"
5 -> "EMERGENCY"
6 -> "SHUTDOWN"
else -> "STATUS_$status"
}
internal fun gameModeName(mode: Int): String = when (mode) {
0 -> "UNSUPPORTED"
1 -> "STANDARD"
2 -> "PERFORMANCE"
3 -> "BATTERY"
4 -> "CUSTOM"
else -> "MODE_$mode"
}
internal fun refreshModesSummary(rates: List<Float>): String =
rates.map { Math.round(it) }.distinct().sortedDescending().joinToString(", ") { "$it Hz" }
internal fun exitSummary(processName: String?, reason: Int, status: Int, description: String?): String =
buildString {
append(processName ?: "?").append(" ").append(exitReasonName(reason)).append(" status=").append(status)
@@ -124,6 +152,19 @@ object DeviceReport {
k("RAM total", FileUtils.sizeToString(mi.totalMem))
k("RAM available", FileUtils.sizeToString(mi.availMem))
}
runCatching {
val pm = context.getSystemService(Context.POWER_SERVICE) as PowerManager
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) k("Thermal status", thermalStatusName(pm.currentThermalStatus))
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
k("Thermal headroom (10 s)", pm.getThermalHeadroom(10).let { if (it.isNaN()) "not reported" else String.format(Locale.US, "%.2f", it) })
}
k("Power saver", pm.isPowerSaveMode)
}
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.S) {
val rate = runCatching { context.getSystemService(PerformanceHintManager::class.java)?.preferredUpdateRateNanos }.getOrNull()
k("ADPF hint sessions", if (rate == null || rate <= 0) "not supported" else "supported (preferred update ${rate / 1000} us)")
k("Game Mode", runCatching { context.getSystemService(android.app.GameManager::class.java)?.gameMode }.getOrNull()?.let { gameModeName(it) })
}
runCatching {
val fs = StatFs(context.filesDir.path)
k("App storage free", FileUtils.sizeToString(fs.availableBlocksLong * fs.blockSizeLong))
@@ -141,6 +182,13 @@ object DeviceReport {
h("Display")
k("Session output", SessionState.outputSize?.let { "${it.first}x${it.second}" })
k("Session refresh", String.format(Locale.US, "%.2f Hz", SessionState.refreshHz))
runCatching {
val display = context.getSystemService(DisplayManager::class.java).getDisplay(Display.DEFAULT_DISPLAY)
k("Panel refresh now", String.format(Locale.US, "%.2f Hz", display.refreshRate))
k("Panel refresh modes", refreshModesSummary(display.supportedModes.map { it.refreshRate }))
}
k("peak_refresh_rate", runCatching { Settings.System.getString(context.contentResolver, "peak_refresh_rate") }.getOrNull() ?: "not set")
k("refresh_rate_mode", runCatching { Settings.Secure.getString(context.contentResolver, "refresh_rate_mode") }.getOrNull() ?: "not set")
val panelSize = com.droiddeck.launcher.session.SessionDisplay.panelSize(context)
for ((label, mode) in listOf("Resolution (Steam)" to SessionService.MODE_STEAM, "Resolution (Desktop)" to SessionService.MODE_DESKTOP)) {
val choice = SessionPrefs.resolutionChoice(context, mode, panelSize)
@@ -1,5 +1,7 @@
package com.droiddeck.launcher.core
import com.droiddeck.launcher.session.SessionPrefs
object GameEnvironment {
data class Config(
val shared: Map<String, String?> = emptyMap(),
@@ -83,4 +85,34 @@ object GameEnvironment {
}
const val CUSTOM_FEX_PRESET = "__custom__"
val syncBackends = listOf(SessionPrefs.SYNC_NTSYNC, SessionPrefs.SYNC_FSYNC, SessionPrefs.SYNC_ESYNC, SessionPrefs.SYNC_WINESERVER)
val syncKeys = listOf("BL_SYNC", "BL_FSYNC_FIRST", "BL_SYNC_FALLBACK", "PROTON_NO_ESYNC", "PROTON_NO_FSYNC", "PROTON_NO_NTSYNC", "WINEESYNC")
fun syncEnvironment(backend: String): Map<String, String> {
require(backend in syncBackends) { "unknown sync $backend" }
return buildMap {
put("BL_SYNC", if (backend == SessionPrefs.SYNC_NTSYNC) "1" else "0")
put("BL_FSYNC_FIRST", if (backend == SessionPrefs.SYNC_FSYNC) "1" else "0")
put("BL_SYNC_FALLBACK", if (backend == SessionPrefs.SYNC_WINESERVER) "0" else "1")
if (backend == SessionPrefs.SYNC_FSYNC) put("PROTON_NO_ESYNC", "1")
}
}
fun gameSync(config: Config, scope: String): String? {
require(scope.isNotEmpty() && validScope(scope))
val entries = config.games[scope].orEmpty().filterKeys { it in syncKeys }
if (entries.isEmpty()) return null
return syncBackends.firstOrNull { entries == syncEnvironment(it) } ?: CUSTOM_SYNC
}
fun withGameSync(config: Config, scope: String, backend: String?): Config {
require(scope.isNotEmpty() && validScope(scope))
val entries = config.entries(scope).filterKeys { it !in syncKeys }.toMutableMap()
if (backend != null) entries += syncEnvironment(backend)
return config.withEntries(scope, entries)
}
const val CUSTOM_SYNC = "__custom__"
}
@@ -51,25 +51,29 @@ object LogRedactor {
@Volatile
private var accounts: List<Learned> = emptyList()
fun learnAccounts(loginUsers: java.io.File) {
val found = ArrayList<Learned>()
try {
if (loginUsers.isFile) {
val kv = Regex("\"(AccountName|PersonaName)\"\\s+\"([^\"]*)\"")
kv.findAll(loginUsers.readText()).map { it.groupValues[2].trim() }.filter { it.length >= 3 }.distinct().forEach { name ->
val needs = name.lowercase().takeIf { name.all { it.code < 0x80 } }
found += Learned(Regex("(?i)(?<![A-Za-z0-9_])" + Regex.escape(name) + "(?![A-Za-z0-9_])"), needs)
}
}
} catch (e: Exception) {
return
fun learnAccounts(vararg loginUsers: java.io.File) {
val names = LinkedHashSet<String>()
val kv = Regex("\"(AccountName|PersonaName)\"\\s+\"([^\"]*)\"")
var unread = false
for (file in loginUsers) {
if (!file.isFile) continue
val text = try { file.readText() } catch (e: Exception) { unread = true; continue }
kv.findAll(text).map { it.groupValues[2].trim() }.filter { it.length >= 3 }.forEach { names += it }
}
accounts = found
val learned = names.map { name ->
val needs = name.lowercase().takeIf { name.all { it.code < 0x80 } }
Learned(Regex("(?i)(?<![A-Za-z0-9_])" + Regex.escape(name) + "(?![A-Za-z0-9_])"), needs)
}
// A file that could not be read keeps the names learned from it before.
accounts = if (unread) (accounts + learned).distinctBy { it.regex.pattern } else learned
}
fun learnFromRuntime(root: java.io.File) {
learnOwnAddresses(java.io.File(root, "etc/droiddeck-net"))
learnAccounts(java.io.File(root, "root/.local/share/Steam/config/loginusers.vdf"))
learnAccounts(
java.io.File(root, "root/.local/share/Steam/config/loginusers.vdf"),
java.io.File(root, "root/.droiddeck-x64/.local/share/Steam/config/loginusers.vdf"),
)
}
@Volatile
@@ -22,7 +22,7 @@ object AddedGameArt {
}
enum class Slot(val key: String, val cacheName: String, val assetType: Int?) {
COVER("p", "p.jpg", 0), BACKGROUND("hero", "hero.jpg", 1), LOGO("logo", "logo.png", 2), ICON("icon", "icon.png", null),
COVER("p", "p.jpg", 0), WIDE("header", "header.jpg", 3), BACKGROUND("hero", "hero.jpg", 1), LOGO("logo", "logo.png", 2), ICON("icon", "icon.png", null),
}
const val FOLDER = "folder"
@@ -40,7 +40,7 @@ object AddedGameArt {
class Found<T>(val items: List<T>, val status: SteamGridDb.Status)
private val CHOSEN = Regex("""chosen-(p|hero|logo|icon)(?:-\d+)?\.(\w+)""")
private val CHOSEN = Regex("""chosen-(p|header|hero|logo|icon)(?:-\d+)?\.(\w+)""")
internal fun chosen(cache: File, slot: Slot): File? =
cache.listFiles { f -> CHOSEN.matchEntire(f.name)?.groupValues?.get(1) == slot.key }?.maxWithOrNull(compareBy({ it.lastModified() }, { it.name }))
@@ -79,7 +79,7 @@ object AddedGameArt {
fun cached(name: String) = File(cache, name).takeIf { it.isFile }
return Art(
portrait = chosen(cache, Slot.COVER) ?: own.portrait ?: cached("p.jpg") ?: exeArt(context, game, cover = true),
header = own.header ?: cached("header.jpg") ?: exeArt(context, game, cover = false),
header = chosen(cache, Slot.WIDE) ?: own.header ?: cached("header.jpg") ?: exeArt(context, game, cover = false),
hero = chosen(cache, Slot.BACKGROUND) ?: own.hero ?: cached("hero.jpg"),
logo = chosen(cache, Slot.LOGO) ?: own.logo ?: cached("logo.png"),
icon = chosen(cache, Slot.ICON) ?: own.icon ?: cached("icon-sgdb.png") ?: exeIcon(context, game),
@@ -96,6 +96,7 @@ object AddedGameArt {
chosen(cache, slot)?.let { return@map SlotState(slot, it, chosenSource(context, game, slot).first.ifEmpty { FILE }, true) }
val auto: Pair<File, String>? = when (slot) {
Slot.COVER -> own.portrait?.let { it to FOLDER } ?: cached("p.jpg") ?: exeArt(context, game, cover = true)?.let { it to EXE }
Slot.WIDE -> own.header?.let { it to FOLDER } ?: cached("header.jpg") ?: exeArt(context, game, cover = false)?.let { it to EXE }
Slot.BACKGROUND -> own.hero?.let { it to FOLDER } ?: cached("hero.jpg")
Slot.LOGO -> own.logo?.let { it to FOLDER } ?: cached("logo.png")
Slot.ICON -> own.icon?.let { it to FOLDER } ?: File(cache, "icon-sgdb.png").takeIf { it.isFile }?.let { it to SGDB }
@@ -124,7 +125,7 @@ object AddedGameArt {
}
internal fun steamOption(steamId: String, slot: Slot): Option? {
val remote = when (slot) { Slot.COVER -> "library_600x900.jpg"; Slot.BACKGROUND -> "library_hero.jpg"; Slot.LOGO -> "logo.png"; Slot.ICON -> null } ?: return null
val remote = when (slot) { Slot.COVER -> "library_600x900.jpg"; Slot.WIDE -> "header.jpg"; Slot.BACKGROUND -> "library_hero.jpg"; Slot.LOGO -> "logo.png"; Slot.ICON -> null } ?: return null
val url = "$CDN$steamId/$remote".takeIf { exists(it) } ?: steamAssetUrl(steamId, remote) ?: return null
return Option(STEAM, url, url)
}
@@ -51,6 +51,16 @@ object AddedGameEdits {
}
}
fun logoPosition(context: Context, folder: String): LogoPosition.Value? =
game(context, folder)?.let { SteamGrid.position(SteamGrid.target(it, context)) }
fun setLogoPosition(context: Context, folder: String, value: LogoPosition.Value?): Boolean {
val game = game(context, folder) ?: return false
if (!SteamGrid.writePosition(context, SteamGrid.target(game, context), value)) return false
publish(context) { SteamLiveShortcuts.setLogoPosition(context, game.appId, value) }
return true
}
private fun applyArt(context: Context, game: AddedGames.Game, slot: AddedGameArt.Slot) {
val file = AddedGameArt.describe(context, game).first { it.slot == slot }.file
if (slot != AddedGameArt.Slot.ICON) SteamGrid.write(context, SteamGrid.target(game, context), slot, AddedGameArt.chosen(AddedGameArt.cacheDir(context, game), slot))
@@ -6,6 +6,7 @@ import com.droiddeck.launcher.runtime.LinuxRuntime
import org.json.JSONObject
import com.droiddeck.launcher.session.GameStorage
import com.droiddeck.launcher.session.SessionPrefs
import com.droiddeck.launcher.stores.Store
import com.droiddeck.launcher.stores.StoreGameSidecar
import com.droiddeck.launcher.stores.StoreInstallRoot
import java.io.File
@@ -138,6 +139,10 @@ object AddedGames {
"appxmanifest.xml" to Regex("<Application\\b[^>]*?\\bExecutable\\s*=\\s*\"([^\"]+)\"", RegexOption.IGNORE_CASE),
)
/** The card's Steam libraries: the arm64 client's steamapps and the x86-64 client's own folder beside it. */
internal fun isSteamLibraryFolder(name: String): Boolean =
name.equals("steamapps", ignoreCase = true) || name == com.droiddeck.launcher.session.GameStorage.X64_CARD_DIR
fun scan(context: Context): List<Game> {
val out = ArrayList<Game>()
GamesFolderMigration.run(context)
@@ -150,7 +155,7 @@ object AddedGames {
if (!dir.isDirectory) { Log.w(TAG, "$dir is not a folder; skipped"); continue }
val steamInstalls = steamInstallDirs(dir)
for (folder in dir.listFiles { f -> f.isDirectory }?.sortedBy { it.name.lowercase() } ?: emptyList()) {
if (folder.name.lowercase() in steamInstalls || folder.name.equals("steamapps", ignoreCase = true)) continue
if (folder.name.lowercase() in steamInstalls || isSteamLibraryFolder(folder.name)) continue
if (folder.path in removed) continue
if (folder.name == "Games" && StoreInstallRoot.roots(context).any { it.absolutePath == folder.absolutePath }) continue
scanGame(context, folder, out)
@@ -210,7 +215,7 @@ object AddedGames {
val base = canonical(dir)
if (!inside(path, base)) continue
val first = path.removePrefix("$base/").substringBefore('/')
File(dir, first).takeIf { it.isDirectory && first != "steamapps" }?.let { return it }
File(dir, first).takeIf { it.isDirectory && !isSteamLibraryFolder(first) }?.let { return it }
}
return null
}
@@ -270,7 +275,7 @@ object AddedGames {
)
for (dir in dirs.distinctBy { it.absolutePath }) {
for (folder in dir.listFiles { f -> f.isDirectory }.orEmpty()) {
if (folder.path in removed || folder.name.equals("steamapps", ignoreCase = true)) continue
if (folder.path in removed || isSteamLibraryFolder(folder.name)) continue
if (folder.name == "Games" && StoreInstallRoot.roots(context).any { it.absolutePath == folder.absolutePath }) continue
out.add(canonical(folder))
}
@@ -296,19 +301,22 @@ object AddedGames {
private fun scanGame(context: Context, folder: File, out: MutableList<Game>, single: AddedExes.Entry? = null) {
run {
val sidecar = StoreGameSidecar.read(folder)
sidecar?.takeIf { it.id.isNotEmpty() }?.let { SessionPrefs.followStoreGame(context, "${it.store.id}:${it.id}", folder.path) { old -> !File(old).exists() } }
val ranking = rank(folder)
val candidates = ranking.exes
val picked = SessionPrefs.addedGameExe(context, folder.path)
val chosen = picked.takeIf { it.isNotEmpty() }?.let { File(it) }?.takeIf { it.isFile }
?: single?.takeIf { it.picked }?.let { File(it.exe) }?.takeIf { it.isFile }
val found = chosen ?: sidecar?.exeFile(folder)?.takeIf { it.isFile } ?: candidates.firstOrNull() ?: return
val launch = if (chosen == null) sidecar?.launcherFile(folder)?.takeIf { it.isFile } ?: found else found
val epicLauncher = sidecar?.takeIf { it.store == Store.EPIC && (chosen == null || canonical(chosen) == canonical(it.exeFile(folder))) }
?.launcherFile(folder)?.takeIf { it.isFile }
val launch = epicLauncher ?: if (chosen == null) sidecar?.launcherFile(folder)?.takeIf { it.isFile } ?: found else found
val foundGuest = guestPath(context, launch)
if (foundGuest == null) { Log.w(TAG, "${folder.name}: the session cannot see ${launch.path}"); return }
val autoName = sidecar?.title ?: identifiedName(folder, found) ?: folder.name
val crc = CRC32().apply { update(("\"${foundGuest.replaceFirst(Regex("^$LIBRARY/"), "$LEGACY_LIBRARY/")}\"" + autoName).toByteArray()) }.value
val appId = SessionPrefs.addedGameAppId(context, folder.path, crc or 0x80000000L)
val config = accountConfig(File(LinuxRuntime.rootDir(context), "root/.local/share/Steam"))
val config = accountConfig(SessionPrefs.activeSteamRoot(context))
val record = config?.let { shortcutRecord(it, appId) }
val rev = record?.optInt("rev", 0) ?: 0
if (sidecar == null && record != null) adoptSteamEdits(context, folder, record)
@@ -343,6 +351,9 @@ object AddedGames {
JSONObject(File(config, ".droiddeck-routes.json").readText()).optInt(appId.toString()).takeIf { it > 0 }
}.getOrNull()
fun inSteam(context: Context, appId: Long): Boolean =
accountConfig(SessionPrefs.activeSteamRoot(context))?.let { shortcutRecord(it, appId) } != null
private fun shortcutRecord(config: File, appId: Long): JSONObject? = runCatching {
JSONObject(File(config, ".droiddeck-shortcuts.json").readText()).optJSONObject("games")?.optJSONObject(appId.toString())
}.getOrNull()
@@ -382,7 +393,8 @@ object AddedGames {
}
}
fun writeListing(context: Context, games: List<Game>): File {
fun writeListing(context: Context, games: List<Game>,
extra: List<JSONObject> = listOfNotNull(com.droiddeck.launcher.stores.epic.EpicClient.steamListing(context))): File {
val file = File(context.filesDir, "session/added-games.json").apply { parentFile?.mkdirs() }
val json = StringBuilder("[")
games.forEachIndexed { i, g ->
@@ -399,6 +411,10 @@ object AddedGames {
if (pieces.isNotEmpty()) json.append(",\"art\":{").append(pieces.joinToString(",") { (k, v) -> quote(k) + ":" + quote(v) }).append('}')
json.append('}')
}
extra.forEachIndexed { i, entry ->
if (games.isNotEmpty() || i > 0) json.append(',')
json.append(entry.toString())
}
file.writeText(json.append(']').toString())
return file
}
@@ -26,6 +26,7 @@ object AddedGamesPublisher {
fun republish(context: Context) {
val games = AddedGames.scan(context)
AddedGames.writeListing(context, games)
LibraryCache.save(context, Library.launchableGames(context, added = games))
val arch = com.droiddeck.launcher.session.SessionPrefs.activeSteamClientArch(context)
LibraryCache.save(context, Library.launchableGames(context, added = games, arch = arch), arch == "x86_64")
}
}
@@ -38,7 +38,7 @@ object GameFileSync {
@Synchronized private fun sync(context: Context, folder: File) {
check(LinuxRuntime.isInstalled(context)) { context.getString(R.string.game_sync_err_runtime) }
check(File(LinuxRuntime.rootDir(context), "root/.local/share/Steam/steamapps").isDirectory) { context.getString(R.string.game_sync_err_steam_library) }
check(File(SessionPrefs.activeSteamRoot(context), "steamapps").isDirectory) { context.getString(R.string.game_sync_err_steam_library) }
val storage = GameStorage.effective(context)
val storageSetting = SessionPrefs.gameStorage(context)
val previousStorage = prefs(context).getString("storagePath", null)
@@ -25,7 +25,7 @@ object GamesFolderMigration {
var n = 0
val steamInstalls = AddedGames.steamInstallDirs(root.host)
for (folder in root.host.listFiles { f -> f.isDirectory }?.sortedBy { it.name.lowercase() }.orEmpty()) {
if (folder.name.equals("steamapps", ignoreCase = true) || folder.name.lowercase() in steamInstalls) continue
if (AddedGames.isSteamLibraryFolder(folder.name) || folder.name.lowercase() in steamInstalls) continue
if (folder.absolutePath in storeRoots || StoreGameSidecar.read(folder) != null) continue
if (folder.path in entries) continue
val picked = SessionPrefs.addedGameExe(context, folder.path).takeIf { it.isNotEmpty() }?.let { File(it) }?.takeIf { it.isFile }
@@ -24,6 +24,8 @@ object Library {
val profileKey: String get() = if (library == ADDED) java.lang.Integer.toUnsignedString(appId) else appId.toString()
}
const val ADDED = "added"
const val EPIC_LAUNCHER = "epic-launcher"
const val EPIC_LAUNCHER_NAME = "Epic Games Launcher"
const val SOURCE_STEAM = "steam"
class Rom(val name: String, val hostPath: File, val guestPath: String, val emulatorId: String, val art: File? = null)
@@ -72,11 +74,14 @@ object Library {
private val LAST_PLAYED = Regex("^\\s*\"LastPlayed\"\\s*\"(\\d+)\"", RegexOption.MULTILINE)
private val INSTALL_DIR = Regex("^\\s*\"installdir\"\\s*\"([^\"]*)\"", RegexOption.MULTILINE)
internal fun steamLibraries(context: Context): List<Pair<File, String>> {
val root = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam")
internal fun steamLibraries(context: Context, arch: String = SessionPrefs.activeSteamClientArch(context)): List<Pair<File, String>> {
val root = SessionPrefs.steamRoot(context, arch)
// The session gives the x86-64 client only its card folder; the games folders stay the arm64 client's.
val card = GameStorage.effective(context)
if (arch == "x86_64") return listOfNotNull(root to "internal", card?.let { File(it.path, GameStorage.X64_CARD_DIR) to it.label })
return listOfNotNull(
root to "internal",
GameStorage.effective(context)?.let { File(it.path) to it.label },
card?.let { File(it.path) to it.label },
) + GameStorage.gamesFolderLibraries(context).map { it.host to it.label }
}
@@ -96,10 +101,10 @@ object Library {
.firstOrNull { it.isDirectory }
}
fun steamGames(context: Context, strictRead: Boolean = false): List<SteamGame> {
val root = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam")
fun steamGames(context: Context, strictRead: Boolean = false, arch: String = SessionPrefs.activeSteamClientArch(context)): List<SteamGame> {
val root = SessionPrefs.steamRoot(context, arch)
val cache = File(root, "appcache/librarycache")
val libraries = steamLibraries(context)
val libraries = steamLibraries(context, arch)
val roots = libraries.map { it.first }
val out = LinkedHashMap<Int, SteamGame>()
for ((library, label) in libraries) {
@@ -132,19 +137,24 @@ object Library {
}
fun steamLibraryArt(context: Context): List<Pair<Int, File>> {
val cache = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam/appcache/librarycache")
val cache = File(SessionPrefs.activeSteamRoot(context), "appcache/librarycache")
return cache.listFiles().orEmpty().mapNotNull { dir ->
val appId = dir.name.toIntOrNull()?.takeIf { it !in NOT_GAMES } ?: return@mapNotNull null
steamCacheImage(cache, appId, STEAM_CAPSULES)?.let { appId to it }
}
}
fun launchableGames(context: Context, strictRead: Boolean = false, added: List<AddedGames.Game>? = null): List<SteamGame> {
val roots = steamLibraries(context).map { it.first }
fun fromSteamClient(g: SteamGame): Boolean = g.library != ADDED && g.source == SOURCE_STEAM
fun launchableGames(
context: Context, strictRead: Boolean = false, added: List<AddedGames.Game>? = null,
arch: String = SessionPrefs.activeSteamClientArch(context),
): List<SteamGame> {
val roots = steamLibraries(context, arch).map { it.first }
val list = added ?: AddedGames.scan(context)
var steam = steamGames(context, strictRead)
val targets = steam.filter { it.library != ADDED && it.source == SOURCE_STEAM }.map { SteamGrid.target(it, context) } + list.map { SteamGrid.target(it, context) }
if (runCatching { SteamGrid.sync(context, targets) }.getOrDefault(false)) steam = steamGames(context, strictRead)
var steam = steamGames(context, strictRead, arch)
val targets = steam.filter(::fromSteamClient).map { SteamGrid.target(it, context) } + list.map { SteamGrid.target(it, context) }
if (runCatching { SteamGrid.sync(context, targets) }.getOrDefault(false)) steam = steamGames(context, strictRead, arch)
return (steam + list.map { g -> addedGame(context, g, roots) }).distinctBy { it.gameId }
}
@@ -192,7 +202,8 @@ object Library {
if (program == com.droiddeck.launcher.runtime.FlatpakManager.LAUNCHER || program == com.droiddeck.launcher.runtime.AppImageManager.LAUNCHER ||
program == com.droiddeck.launcher.runtime.UserApps.SCRIPT_LAUNCHER) {
com.droiddeck.launcher.session.SessionState.programArgs.firstOrNull()?.let { flatpakNames[it] ?: it.substringAfterLast('.') }
} else specs.firstOrNull { it.program == program }?.name
} else com.droiddeck.launcher.session.NoSteamLaunch.nameFor(program, com.droiddeck.launcher.session.SessionState.programArgs)
?: specs.firstOrNull { it.program == program }?.name
val flatpakNames = java.util.concurrent.ConcurrentHashMap<String, String>()
@@ -10,10 +10,14 @@ object LibraryCache {
private const val TAG = "LibraryCache"
private fun file(context: Context) = File(context.filesDir, "library-cache.json")
fun load(context: Context): List<Library.SteamGame> = runCatching {
/** The cached games and whether they are the x86-64 client's; a cache written as a bare list is arm64's. */
@Synchronized
fun snapshot(context: Context): Pair<List<Library.SteamGame>, Boolean> = runCatching {
val f = file(context)
if (!f.isFile) return emptyList()
val array = JSONArray(f.readText())
if (!f.isFile) return Pair(emptyList(), false)
val text = f.readText().trim()
val (array, x64) = if (text.startsWith("{")) JSONObject(text).let { it.getJSONArray("games") to (it.optString("client") == "x86_64") }
else JSONArray(text) to false
List(array.length()) { i ->
val o = array.getJSONObject(i)
Library.SteamGame(
@@ -24,11 +28,18 @@ object LibraryCache {
source = o.optString("source").ifEmpty { if (o.getString("library") == Library.ADDED) Library.ADDED else Library.SOURCE_STEAM },
storeId = o.optString("storeId").ifEmpty { null },
)
}
}.getOrElse { e -> Log.w(TAG, "unreadable, ignored: ${e.message}"); emptyList() }
} to x64
}.getOrElse { e -> Log.w(TAG, "unreadable, ignored: ${e.message}"); Pair(emptyList(), false) }
fun save(context: Context, games: List<Library.SteamGame>) {
fun load(context: Context): List<Library.SteamGame> = snapshot(context).first
fun listedX64(context: Context): Boolean = snapshot(context).second
/** Without [x64] the list keeps the client already recorded. */
@Synchronized
fun save(context: Context, games: List<Library.SteamGame>, x64: Boolean? = null) {
runCatching {
val client = x64 ?: listedX64(context)
val array = JSONArray()
for (g in games) array.put(
JSONObject().put("appId", g.appId).put("name", g.name).put("library", g.library).put("gameId", g.gameId)
@@ -38,8 +49,12 @@ object LibraryCache {
)
val f = file(context)
val tmp = File(f.path + ".tmp")
tmp.writeText(array.toString())
tmp.renameTo(f)
tmp.writeText(JSONObject().put("client", if (client) "x86_64" else "arm64").put("games", array).toString())
if (!tmp.renameTo(f)) {
tmp.delete()
f.delete()
Log.w(TAG, "not saved: rename failed; cache dropped so the next start scans")
}
}.onFailure { e -> Log.w(TAG, "not saved: ${e.message}") }
}
@@ -0,0 +1,48 @@
package com.droiddeck.launcher.frontend
import org.json.JSONObject
/** Where Steam pins a game's logo on its hero image and how big it draws it; the same JSON Steam keeps in `<appid>.json`. */
object LogoPosition {
enum class Pin(val steam: String) {
UPPER_LEFT("UpperLeft"), UPPER_CENTER("UpperCenter"), CENTER("CenterCenter"), BOTTOM_LEFT("BottomLeft"), BOTTOM_CENTER("BottomCenter");
companion object {
fun of(steam: String): Pin? = values().firstOrNull { it.steam == steam }
}
}
data class Value(val pin: Pin, val widthPct: Int, val heightPct: Int)
const val MIN_PCT = 10
const val MAX_PCT = 100
/** What Steam draws when a game has no custom position. */
val DEFAULT = Value(Pin.BOTTOM_LEFT, 50, 50)
private const val VERSION = 1
private const val KEY = "logoPosition"
fun sized(pin: Pin, size: Int): Value = size.coerceIn(MIN_PCT, MAX_PCT).let { Value(pin, it, it) }
/** `{"nVersion":1,"logoPosition":{...}}`, with whatever else [into] already held kept as it was. */
fun toJson(value: Value, into: JSONObject = JSONObject()): JSONObject = into
.put("nVersion", VERSION)
.put(KEY, JSONObject().put("pinnedPosition", value.pin.steam).put("nWidthPct", value.widthPct).put("nHeightPct", value.heightPct))
fun parse(text: String): Value? = runCatching { parse(JSONObject(text)) }.getOrNull()
fun parse(json: JSONObject): Value? {
val inner = json.optJSONObject(KEY) ?: return null
val pin = Pin.of(inner.optString("pinnedPosition")) ?: return null
if (!inner.has("nWidthPct") || !inner.has("nHeightPct")) return null
return Value(pin, inner.optInt("nWidthPct").coerceIn(MIN_PCT, MAX_PCT), inner.optInt("nHeightPct").coerceIn(MIN_PCT, MAX_PCT))
}
/** [existing] without the logo position; null when nothing but the version number would be left. */
fun without(existing: JSONObject): JSONObject? {
existing.remove(KEY)
val rest = existing.keys().asSequence().filter { it != "nVersion" }.toList()
return if (rest.isEmpty()) null else existing
}
}
@@ -25,9 +25,16 @@ object SessionInstalls {
STATE.find(text)?.groupValues?.get(1)?.toIntOrNull() == 4, file.parentFile!!)
}.getOrNull()
internal fun steamappsDirs(context: Context): List<File> = listOfNotNull(
File(LinuxRuntime.rootDir(context), "root/.local/share/Steam/steamapps"),
GameStorage.effective(context)?.let { File(it.path, "steamapps") },
internal fun steamappsDirs(context: Context): List<File> = (
if (com.droiddeck.launcher.session.SessionPrefs.activeSteamClientArch(context) == "x86_64")
listOfNotNull(
File(com.droiddeck.launcher.session.SessionPrefs.activeSteamRoot(context), "steamapps"),
GameStorage.x64CardLibrary(context)?.let { File(it, "steamapps") },
)
else listOfNotNull(
File(LinuxRuntime.rootDir(context), "root/.local/share/Steam/steamapps"),
GameStorage.effective(context)?.let { File(it.path, "steamapps") },
)
).filter { it.isDirectory }
internal class Watch(private val dirs: () -> List<File>) {
@@ -3,7 +3,6 @@ package com.droiddeck.launcher.frontend
import android.content.Context
import android.util.AtomicFile
import android.util.Log
import com.droiddeck.launcher.runtime.LinuxRuntime
import org.json.JSONArray
import org.json.JSONObject
import java.io.DataInputStream
@@ -20,7 +19,7 @@ object SteamAppInfo {
class Info(val name: String, val sharedDepots: List<String>)
fun appinfoFile(context: Context) = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam/appcache/appinfo.vdf")
fun appinfoFile(context: Context) = File(com.droiddeck.launcher.session.SessionPrefs.activeSteamRoot(context), "appcache/appinfo.vdf")
@Synchronized
fun info(context: Context, appId: Int, appinfo: File = appinfoFile(context)): Info? {
@@ -1,8 +1,8 @@
package com.droiddeck.launcher.frontend
import android.content.Context
import com.droiddeck.launcher.runtime.LinuxRuntime
import com.droiddeck.launcher.stores.SteamLiveShortcuts
import com.droiddeck.launcher.session.SessionPrefs
import java.io.File
object SteamGameArt {
@@ -35,7 +35,7 @@ object SteamGameArt {
}
fun describe(context: Context, game: Library.SteamGame): List<AddedGameArt.SlotState> {
val cache = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam/appcache/librarycache")
val cache = File(SessionPrefs.activeSteamRoot(context), "appcache/librarycache")
val own = withOverrides(context, game)
return AddedGameArt.Slot.values().map { slot ->
chosenFile(context, game.appId, slot)?.let {
@@ -43,6 +43,7 @@ object SteamGameArt {
}
val auto = when (slot) {
AddedGameArt.Slot.COVER -> own.art
AddedGameArt.Slot.WIDE -> Library.steamCacheImage(cache, game.appId, listOf("library_header.jpg", "header.jpg"))
AddedGameArt.Slot.BACKGROUND -> own.hero
AddedGameArt.Slot.ICON -> own.icon
AddedGameArt.Slot.LOGO -> Library.steamCacheImage(cache, game.appId, listOf("logo.png"))
@@ -81,6 +82,14 @@ object SteamGameArt {
}
}
fun position(context: Context, game: Library.SteamGame): LogoPosition.Value? = SteamGrid.position(SteamGrid.target(game, context))
fun setPosition(context: Context, game: Library.SteamGame, value: LogoPosition.Value?): Boolean {
if (!SteamGrid.writePosition(context, SteamGrid.target(game, context), value)) return false
AddedGamesPublisher.later(context) { SteamLiveShortcuts.setLogoPosition(context, game.appId.toLong(), value) }
return true
}
private fun push(context: Context, game: Library.SteamGame, slot: AddedGameArt.Slot, file: File?): Boolean {
val type = slot.assetType ?: return true
if (!SteamGrid.write(context, SteamGrid.target(game, context), slot, file)) return false
@@ -2,7 +2,6 @@ package com.droiddeck.launcher.frontend
import android.content.Context
import android.util.Log
import com.droiddeck.launcher.runtime.LinuxRuntime
import com.droiddeck.launcher.session.SessionPrefs
import org.json.JSONObject
import java.io.File
@@ -14,7 +13,7 @@ object SteamGrid {
/** Every change to a game's art, here or in a scan, takes this first so two of them never interleave. */
internal val lock = Any()
private const val MANIFEST = ".droiddeck-art.json"
private val SUFFIX = mapOf(AddedGameArt.Slot.COVER to "p", AddedGameArt.Slot.BACKGROUND to "_hero", AddedGameArt.Slot.LOGO to "_logo")
private val SUFFIX = mapOf(AddedGameArt.Slot.COVER to "p", AddedGameArt.Slot.WIDE to "", AddedGameArt.Slot.BACKGROUND to "_hero", AddedGameArt.Slot.LOGO to "_logo")
private val EXTS = listOf("png", "jpg", "jpeg", "webp")
class Target(val artDir: File, val id: Long, val setSource: (AddedGameArt.Slot, String) -> Unit)
@@ -28,8 +27,7 @@ object SteamGrid {
}
internal fun dir(context: Context): File? {
val root = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam")
return AddedGames.accountConfig(root)?.let { File(it, "grid") }
return AddedGames.accountConfig(SessionPrefs.activeSteamRoot(context))?.let { File(it, "grid") }
}
private fun files(grid: File, id: Long, slot: AddedGameArt.Slot): List<File> =
@@ -37,13 +35,18 @@ object SteamGrid {
private fun signature(file: File?) = if (file == null) "" else "${file.length()}:${file.lastModified()}"
private fun sigFile(target: Target, slot: AddedGameArt.Slot) = File(target.artDir, "grid-${slot.key}")
/** Each Steam client has its own grid folder, so what was last seen there is kept per client. */
private fun sigName(context: Context, key: String) =
if (SessionPrefs.activeSteamClientArch(context) == "x86_64") "grid-x86_64-$key" else "grid-$key"
private fun recorded(target: Target, slot: AddedGameArt.Slot): String? = sigFile(target, slot).takeIf { it.isFile }?.readText()
private fun sigFile(context: Context, target: Target, slot: AddedGameArt.Slot) = File(target.artDir, sigName(context, slot.key))
private fun record(target: Target, slot: AddedGameArt.Slot, value: String) {
private fun recorded(context: Context, target: Target, slot: AddedGameArt.Slot): String? =
sigFile(context, target, slot).takeIf { it.isFile }?.readText()
private fun record(context: Context, target: Target, slot: AddedGameArt.Slot, value: String) {
target.artDir.mkdirs()
sigFile(target, slot).writeText(value)
sigFile(context, target, slot).writeText(value)
}
private fun writtenByDroidDeck(grid: File): Set<String> = runCatching {
@@ -59,7 +62,7 @@ object SteamGrid {
val grid = dir(context) ?: return true
if (file == null) {
if (files(grid, target.id, slot).map { it.delete() }.any { !it }) return false
record(target, slot, "")
record(context, target, slot, "")
return true
}
val ext = file.extension.lowercase().let { if (it == "jpeg") "jpg" else it }
@@ -73,10 +76,90 @@ object SteamGrid {
return false
}
if (files(grid, target.id, slot).filter { it != dst }.map { it.delete() }.any { !it }) return false
record(target, slot, signature(placed))
record(context, target, slot, signature(placed))
return true
}
private const val POSITION = "logo-position"
private const val POSITION_SIG = "json"
private fun positionFile(grid: File, id: Long) = File(grid, "$id.json")
private fun positionIn(file: File): LogoPosition.Value? = file.takeIf { it.isFile }?.let { runCatching { it.readText() }.getOrNull() }?.let(LogoPosition::parse)
/** The logo position chosen here (or taken over from Steam) for a game; null when Steam's own default applies. */
fun position(target: Target): LogoPosition.Value? = positionIn(File(target.artDir, POSITION))
private fun positionRecorded(context: Context, target: Target): String? =
File(target.artDir, sigName(context, POSITION_SIG)).takeIf { it.isFile }?.readText()
private fun recordPosition(context: Context, target: Target, value: String) {
target.artDir.mkdirs()
File(target.artDir, sigName(context, POSITION_SIG)).writeText(value)
}
private fun storePosition(target: Target, value: LogoPosition.Value?): Boolean {
val file = File(target.artDir, POSITION)
if (value == null) return !file.exists() || file.delete()
target.artDir.mkdirs()
return runCatching { file.writeText(LogoPosition.toJson(value).toString()) }.isSuccess
}
/** Sets (or with null, clears) a game's logo position here and in the client's grid folder, whether or not Steam is running. */
fun writePosition(context: Context, target: Target, value: LogoPosition.Value?): Boolean = synchronized(lock) {
placePosition(context, target, value) && storePosition(target, value)
}
/** Steam keeps the position in `<id>.json`; anything else that file holds is left as it was. */
private fun placePosition(context: Context, target: Target, value: LogoPosition.Value?): Boolean {
val grid = dir(context) ?: return true
val file = positionFile(grid, target.id)
val existing = if (file.isFile) runCatching { JSONObject(file.readText()) }.getOrNull() else null
if (file.isFile && existing == null) return value == null
val next = if (value == null) existing?.let(LogoPosition::without) else LogoPosition.toJson(value, existing ?: JSONObject())
if (next == null) {
if (file.isFile && !file.delete()) return false
recordPosition(context, target, "")
return true
}
grid.mkdirs()
val tmp = File(grid, file.name + ".tmp")
val placed = runCatching { tmp.writeText(next.toString()); tmp.renameTo(file) }.getOrDefault(false)
if (!placed) {
tmp.delete()
return false
}
recordPosition(context, target, signature(file))
return true
}
/** A `logoPosition` this version cannot read (a pin it does not know, say) is left alone rather than read as "removed". */
private fun hasUnknownPosition(file: File): Boolean {
val json = runCatching { JSONObject(file.readText()) }.getOrNull() ?: return false
return json.has("logoPosition") && LogoPosition.parse(json) == null
}
private fun reconcilePosition(context: Context, grid: File, target: Target) {
val file = positionFile(grid, target.id)
if (hasUnknownPosition(file)) return
val theirs = positionIn(file)
val now = if (theirs != null) signature(file) else ""
val before = positionRecorded(context, target)
val own = position(target)
when {
before == null && own != null && theirs == null -> placePosition(context, target, own)
before == null || now != before -> {
// Recorded only once the local side agrees, so a failed take-over is tried again on the next scan.
if (theirs != null) {
if (own == theirs || storePosition(target, theirs)) recordPosition(context, target, now)
} else {
if (own == null || storePosition(target, null)) recordPosition(context, target, "")
}
}
own != null && own != theirs -> placePosition(context, target, own)
}
}
/** Takes over what changed in Steam since the last look; returns true when a choice was added, replaced or dropped. */
fun sync(context: Context, targets: List<Target>): Boolean = synchronized(lock) { reconcile(context, targets) }
@@ -90,20 +173,20 @@ object SteamGrid {
val present = files(grid, target.id, slot)
val theirs = present.filter { it.name !in ours }.maxByOrNull { it.lastModified() }
val now = signature(theirs)
val before = recorded(target, slot)
val before = recorded(context, target, slot)
when {
before == null && own != null && theirs == null -> write(context, target, slot, own)
before == null || now != before -> {
if (theirs != null) {
if (own != null && own.length() == theirs.length() && own.readBytes().contentEquals(theirs.readBytes())) record(target, slot, now)
if (own != null && own.length() == theirs.length() && own.readBytes().contentEquals(theirs.readBytes())) record(context, target, slot, now)
else if (AddedGameArt.store(target.artDir, slot, AddedGameArt.Option(AddedGameArt.STEAM, theirs.path, theirs.path))) {
target.setSource(slot, AddedGameArt.STEAM + "|steam-client")
record(target, slot, now)
record(context, target, slot, now)
changed = true
}
} else {
if (own != null && present.isEmpty()) { own.delete(); target.setSource(slot, ""); changed = true }
record(target, slot, "")
record(context, target, slot, "")
}
}
own != null && theirs != null && now == before && own.length() != theirs.length() -> write(context, target, slot, own)
@@ -113,6 +196,13 @@ object SteamGrid {
Log.w(TAG, "${target.id} ${slot.key}: ${e.javaClass.simpleName}")
}
}
for (target in targets) {
try {
reconcilePosition(context, grid, target)
} catch (e: Exception) {
Log.w(TAG, "${target.id} logo position: ${e.javaClass.simpleName}")
}
}
if (changed) Log.i(TAG, "took over art changed in Steam")
return changed
}
@@ -121,6 +121,7 @@ object SteamGridDb {
internal fun path(id: Int, slot: AddedGameArt.Slot): String =
when (slot) {
AddedGameArt.Slot.COVER -> "grids/game/$id?dimensions=600x900&$QUERY"
AddedGameArt.Slot.WIDE -> "grids/game/$id?dimensions=460x215,920x430&$QUERY"
AddedGameArt.Slot.BACKGROUND -> "heroes/game/$id?$QUERY"
AddedGameArt.Slot.LOGO -> "logos/game/$id?$QUERY"
AddedGameArt.Slot.ICON -> "icons/game/$id?$QUERY"
@@ -0,0 +1,862 @@
package com.droiddeck.launcher.glasses
import android.annotation.SuppressLint
import android.app.PendingIntent
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.hardware.display.DisplayManager
import android.hardware.usb.UsbDevice
import android.hardware.usb.UsbManager
import android.os.Build
import android.os.Handler
import android.os.HandlerThread
import android.os.SystemClock
import android.util.Log
import android.view.Display
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import org.json.JSONArray
import org.json.JSONObject
import kotlin.math.abs
import kotlin.math.sin
import com.droiddeck.launcher.R
import java.io.File
data class GlassesState(
val present: Boolean = false,
val model: String = "VITURE",
val controllable: Boolean = false,
val permission: Glasses.Permission = Glasses.Permission.NONE,
val connected: Boolean = false,
val displayId: Int = Display.INVALID_DISPLAY,
val width: Int = 0,
val height: Int = 0,
val hz: Float = 0f,
val sbs: Boolean = false,
val switching3d: Boolean = false,
val brightness: Int = -1,
val volume: Int = -1,
val brightnessMax: Int = 8,
val volumeMax: Int = 8,
val sunblock: Boolean = false,
val hardwareControls: Boolean = true,
val hasSunblock: Boolean = true,
val controlError: String? = null,
val tracking: Boolean = false,
val diagonalFovDeg: Float = 46f,
val noSignal: Boolean = false,
val sbsSupport: Int = SBS_UNKNOWN,
val sbs90: Boolean = false,
val experimental: Boolean = false,
) {
val hasDisplay: Boolean get() = displayId != Display.INVALID_DISPLAY
val usable: Boolean get() = hasDisplay && !noSignal
val canControl: Boolean get() = connected && controllable
val can3d: Boolean get() = canControl && !noSignal && sbsSupport != SBS_FAILS
val verticalFovDeg: Float get() {
val half = Math.toRadians(diagonalFovDeg / 2.0)
val aspect = if (width > 0 && height > 0)
width.toDouble() / (if (sbs) 2 else 1) / height else 16.0 / 9.0
val v = kotlin.math.atan(kotlin.math.tan(half) / kotlin.math.sqrt(1 + aspect * aspect))
return Math.toDegrees(2 * v).toFloat()
}
companion object {
const val SBS_UNKNOWN = 0
const val SBS_WORKS = 1
const val SBS_FAILS = -1
}
}
data class GlassesSettings(
val refreshHz: Int = 120,
val sbsHz: Int = 60,
val sunblock: Boolean? = null,
val gameOnGlasses: Boolean = true,
val headTracking: Boolean = true,
val auto3d: Boolean = false,
val picture3d: Int = PICTURE_AUTO,
val screenWidthDm: Int = 20,
val screenDistanceDm: Int = 30,
val vrFovDeg: Int = VR_FOV_DEFAULT,
) {
companion object {
const val VR_FOV_MIN = 24
const val VR_FOV_MAX = 90
const val VR_FOV_DEFAULT = 90
const val PICTURE_AUTO = 0
const val PICTURE_SCREEN = 1
const val PICTURE_SBS = 2
}
}
@SuppressLint("StaticFieldLeak")
object Glasses {
enum class Permission { NONE, ASKING, DENIED, GRANTED }
fun interface PoseSink {
fun onPose(x: Float, y: Float, z: Float, w: Float, timeNs: Long)
}
private const val TAG = "Glasses"
private const val ACTION_PERMISSION = "com.droiddeck.launcher.glasses.USB_PERMISSION"
private const val PREFS = "glasses"
private const val KEY_SBS_FAILED = "sbsFailed"
private const val SBS_CONFIRM_MS = 20_000L
private val _state = MutableStateFlow(GlassesState())
val state: StateFlow<GlassesState> = _state.asStateFlow()
private val _settings = MutableStateFlow(GlassesSettings())
val settings: StateFlow<GlassesSettings> = _settings.asStateFlow()
private val _calibration = MutableStateFlow(GlassesCalibrationState())
val calibration: StateFlow<GlassesCalibrationState> = _calibration.asStateFlow()
private lateinit var calibrationStore: RayNeoCalibrationStore
private var calibrationKey: String? = null
private var calibrationRevision = 0L
private lateinit var app: Context
private lateinit var handler: Handler
private lateinit var usb: UsbManager
private lateinit var displays: DisplayManager
@Volatile private var link: GlassesLink? = null
@Volatile private var linkGeneration = 0L
private var linkDevice: UsbDevice? = null
private var model: GlassesModel? = null
private var asked = false
private var lastModeSentAt = 0L
private val modeState = GlassesModeState(SBS_CONFIRM_MS)
private val confirmMode = Runnable { refresh() }
private var glassesMode = 0
private var autoModeSent = false
@Volatile private var sink: PoseSink? = null
@Volatile private var probe: ((ByteArray?, FloatArray?) -> Unit)? = null
@Volatile private var latest: FloatArray? = null
@Volatile private var imuPackets = 0L
private val simulating = java.util.concurrent.atomic.AtomicReference<Thread?>(null)
private val frames = ArrayDeque<String>()
fun init(context: Context) {
if (::handler.isInitialized) return
app = context.applicationContext
usb = app.getSystemService(Context.USB_SERVICE) as UsbManager
displays = app.getSystemService(Context.DISPLAY_SERVICE) as DisplayManager
_settings.value = load()
calibrationStore = RayNeoCalibrationStore(app.getSharedPreferences(PREFS, Context.MODE_PRIVATE))
handler = Handler(HandlerThread("droiddeck-glasses").apply { start() }.looper)
handler.post {
val filter = IntentFilter(ACTION_PERMISSION).apply {
addAction(UsbManager.ACTION_USB_DEVICE_ATTACHED)
addAction(UsbManager.ACTION_USB_DEVICE_DETACHED)
}
if (Build.VERSION.SDK_INT >= 33) app.registerReceiver(receiver, filter, null, handler, Context.RECEIVER_NOT_EXPORTED)
else app.registerReceiver(receiver, filter, null, handler)
displays.registerDisplayListener(displayListener, handler)
refresh()
}
}
fun rescan() = post { refresh() }
fun startCalibration() = post {
val l = link as? RayNeoLink ?: return@post
if (!l.ready || _calibration.value.running || simulating.get() != null) return@post
_calibration.value = _calibration.value.copy(phase = CalibrationPhase.SETTLING, progress = 0)
l.startCalibration(++calibrationRevision)
updateImu()
}
fun cancelCalibration() = post {
(link as? RayNeoLink)?.cancelCalibration(++calibrationRevision, calibrationKey?.let(calibrationStore::load))
_calibration.value = _calibration.value.copy(phase = CalibrationPhase.IDLE, progress = 0)
updateImu()
}
fun resetCalibration() = post {
(link as? RayNeoLink)?.resetCalibration(++calibrationRevision)
calibrationKey?.let { calibrationStore.save(it, null) }
_calibration.value = _calibration.value.copy(saved = false, phase = CalibrationPhase.IDLE, progress = 0)
updateImu()
}
fun query() = post {
val l = link ?: return@post
listOf(VitureProtocol.MSG_GET_DISPLAY, VitureProtocol.MSG_GET_BRIGHTNESS, VitureProtocol.MSG_GET_VOLUME, VitureProtocol.MSG_GET_IMU).forEachIndexed { i, msg ->
handler.postDelayed({ if (link === l) l.send(msg) }, VitureLink.COMMAND_GAP_MS * i)
}
}
fun sendMode(mode: Int) = post {
if (!_state.value.canControl) return@post
if (link?.send(VitureProtocol.MSG_DISPLAY, byteArrayOf(mode.toByte())) == true) lastModeSentAt = SystemClock.uptimeMillis()
}
fun requestControl() = post {
asked = false
refresh(ask = true)
}
fun setRefreshHz(hz: Int) = post {
update { it.copy(refreshHz = hz.coerceIn(60, 120)) }
if (!_state.value.sbs) applyMode(force = true)
}
fun setSbsHz(hz: Int) = post {
update { it.copy(sbsHz = if (hz >= 90) 90 else 60) }
if (_state.value.sbs && _state.value.can3d) sendMode(true)
}
fun set3d(on: Boolean) = post {
val s = _state.value
if (!s.canControl || (on && s.noSignal)) return@post
modeState.manual()
if (on) {
markSbsFailed(linkDevice?.productId, false)
rememberSbs(GlassesState.SBS_UNKNOWN)
}
sendMode(on)
}
fun set3dForVr(on: Boolean) = post {
val s = _state.value
if (on) {
if (!s.can3d || s.sbs || modeState.pendingStereo) return@post
if (sendMode(true)) modeState.startVr()
} else if (modeState.stopVr() && s.canControl) {
sendMode(false)
}
}
fun setBrightness(level: Int) = post {
val s = _state.value
if (!s.canControl || s.brightnessMax == 0) return@post
val v = level.coerceIn(0, s.brightnessMax)
if (link?.send(VitureProtocol.MSG_BRIGHTNESS, byteArrayOf(v.toByte(), 0)) == true && link !is XrealLink)
publish { it.copy(brightness = v) }
}
fun setVolume(level: Int) = post {
val s = _state.value
if (!s.canControl || s.volumeMax == 0) return@post
val v = level.coerceIn(0, s.volumeMax)
if (link?.send(VitureProtocol.MSG_VOLUME, byteArrayOf(v.toByte(), 0)) == true) publish { it.copy(volume = v) }
}
fun setSunblock(on: Boolean) = post {
update { it.copy(sunblock = on) }
sendFilm(on)
}
fun setSbsSupport(support: Int) = post {
val s = support.coerceIn(GlassesState.SBS_FAILS, GlassesState.SBS_WORKS)
markSbsFailed(linkDevice?.productId, s == GlassesState.SBS_FAILS)
rememberSbs(s)
}
fun setGameOnGlasses(on: Boolean) = post { update { it.copy(gameOnGlasses = on) } }
fun setHeadTracking(on: Boolean) = post { update { it.copy(headTracking = on) } }
fun setAuto3d(on: Boolean) = post { update { it.copy(auto3d = on) } }
fun setPicture3d(picture: Int) = post { update { it.copy(picture3d = picture.coerceIn(0, 2)) } }
fun setScreenWidthDm(dm: Int) = post { update { it.copy(screenWidthDm = dm.coerceIn(5, 200)) } }
fun setScreenDistanceDm(dm: Int) = post { update { it.copy(screenDistanceDm = dm.coerceIn(5, 200)) } }
fun setVrFovDeg(deg: Int) = post { update { it.copy(vrFovDeg = deg.coerceIn(GlassesSettings.VR_FOV_MIN, GlassesSettings.VR_FOV_MAX)) } }
fun outputHz(): Int = GlassesRendering.outputHz(_settings.value, _state.value)
fun glassesDisplay(): Display? = _state.value.displayId.takeIf { it != Display.INVALID_DISPLAY }?.let { displays.getDisplay(it) }
fun setPoseSink(s: PoseSink) {
synchronized(this) { sink = s }
post { updateImu() }
}
fun clearPoseSink(s: PoseSink) {
synchronized(this) { if (sink === s) sink = null }
post { updateImu() }
}
fun simulate(seconds: Int, holdYaw: Double? = null, holdPitch: Double? = null): Boolean {
val hold = holdYaw != null || holdPitch != null
val t = Thread({
val start = SystemClock.elapsedRealtimeNanos()
val end = start + seconds * 1_000_000_000L
while (SystemClock.elapsedRealtimeNanos() < end && simulating.get() === Thread.currentThread()) {
val now = SystemClock.elapsedRealtimeNanos()
val t = (now - start) / 1e9
val yaw = if (hold) holdYaw ?: 0.0 else 40.0 * sin(t * 2 * Math.PI / 6.0)
val pitch = if (hold) holdPitch ?: 0.0 else 15.0 * sin(t * 2 * Math.PI / 4.0)
val roll = if (hold) 0.0 else 5.0 * sin(t * 2 * Math.PI / 5.0)
val q = VitureProtocol.eulerZxy(roll.toFloat(), pitch.toFloat(), yaw.toFloat())
deliver(q, now)
SystemClock.sleep(4)
}
simulating.compareAndSet(Thread.currentThread(), null)
}, "glasses-simulate")
if (!simulating.compareAndSet(null, t)) return false
t.start()
return true
}
fun stopSimulation() {
simulating.set(null)
}
fun status(): JSONObject {
val s = _state.value
val g = _settings.value
val q = latest
return JSONObject()
.put("present", s.present).put("model", s.model).put("controllable", s.controllable)
.put("permission", s.permission.name).put("connected", s.connected)
.put("controlError", s.controlError ?: JSONObject.NULL)
.put("experimental", s.experimental)
.put("display", if (s.hasDisplay) JSONObject().put("id", s.displayId).put("width", s.width).put("height", s.height).put("hz", s.hz.toDouble()) else JSONObject.NULL)
.put("sbs", s.sbs).put("deviceMode", glassesMode).put("switching3d", s.switching3d)
.put("brightness", s.brightness).put("volume", s.volume)
.put("brightnessMax", s.brightnessMax).put("volumeMax", s.volumeMax).put("sunblock", s.sunblock)
.put("tracking", s.tracking).put("simulating", simulating.get() != null).put("imuPackets", imuPackets)
.put("calibration", JSONObject().put("supported", _calibration.value.supported)
.put("saved", _calibration.value.saved).put("phase", _calibration.value.phase.name)
.put("progress", _calibration.value.progress))
.put("badFrames", link?.badFrames ?: 0L)
.put("pose", q?.let { JSONArray().put(it[0].toDouble()).put(it[1].toDouble()).put(it[2].toDouble()).put(it[3].toDouble()) } ?: JSONObject.NULL)
.put("fovDeg", s.diagonalFovDeg.toDouble()).put("noSignal", s.noSignal).put("sbsSupport", s.sbsSupport)
.put("settings", JSONObject().put("refreshHz", g.refreshHz).put("sbsHz", g.sbsHz).put("sunblock", g.sunblock ?: JSONObject.NULL)
.put("gameOnGlasses", g.gameOnGlasses).put("headTracking", g.headTracking).put("auto3d", g.auto3d)
.put("picture3d", g.picture3d).put("screenWidthDm", g.screenWidthDm).put("screenDistanceDm", g.screenDistanceDm)
.put("vrFovDeg", g.vrFovDeg))
}
fun recentFrames(): JSONArray = synchronized(frames) { JSONArray(frames.toList()) }
fun sendRaw(msgId: Int, payload: ByteArray): Boolean {
val l = link ?: return false
return l.send(msgId, payload)
}
private fun post(block: () -> Unit) {
if (!::handler.isInitialized) return
handler.post(block)
}
private fun publish(transform: (GlassesState) -> GlassesState) {
_state.value = transform(_state.value)
}
private fun update(transform: (GlassesSettings) -> GlassesSettings) {
val next = transform(_settings.value)
_settings.value = next
save(next)
}
private val receiver = object : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
when (intent.action) {
ACTION_PERMISSION -> {
val granted = intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)
Log.i(TAG, "USB permission ${if (granted) "granted" else "denied"}")
if (!granted) publish { it.copy(permission = Permission.DENIED) }
refresh()
}
UsbManager.ACTION_USB_DEVICE_ATTACHED -> {
if (intent.device()?.let { isGlassesVendor(it.vendorId) } == true) asked = false
refresh(ask = true)
}
UsbManager.ACTION_USB_DEVICE_DETACHED -> if (intent.device()?.let { isGlassesVendor(it.vendorId) } == true) {
refresh()
}
}
}
}
private val displayListener = object : DisplayManager.DisplayListener {
override fun onDisplayAdded(displayId: Int) = refresh()
override fun onDisplayRemoved(displayId: Int) = refresh()
override fun onDisplayChanged(displayId: Int) {
if (displayId == _state.value.displayId || displayId != Display.DEFAULT_DISPLAY) refresh()
}
}
private fun refresh(ask: Boolean = false) {
val devices = usb.deviceList.values.filter {
it.vendorId == VitureProtocol.VENDOR_ID || deviceModel(it) != null
}
val known = devices.mapNotNull { device ->
deviceModel(device)?.let { device to it }
}
val selected = known.firstOrNull { canOpenModel(it.second) } ?: known.firstOrNull()
val device = selected?.first
val m = selected?.second
val controllable = m?.let(::canOpenModel) == true
// XREAL uses Android's Ethernet sockets, without opening or claiming USB interfaces.
val permitted = controllable && device != null && (m === XrealModel || usb.hasPermission(device))
val identity = device?.let {
GlassesModeState.Device(it.productId, if (permitted) runCatching { it.serialNumber }.getOrNull() else null)
}
if (modeState.updateDevice(identity, SystemClock.uptimeMillis())) {
rememberSbs(if (identity != null && sbsFailed(identity.productId)) GlassesState.SBS_FAILS else GlassesState.SBS_UNKNOWN)
}
if (device == null || (linkDevice != null && linkDevice?.deviceName != device.deviceName)) closeLink()
model = m
val display = findDisplay(devices.isNotEmpty())
val mode = display?.mode
val width = mode?.physicalWidth ?: 0
val height = mode?.physicalHeight ?: 0
val sbsByDisplay = width >= 3000
val noSignal = display != null && width in 1 until 1280
when (modeState.observe(sbsByDisplay, SystemClock.uptimeMillis())) {
GlassesModeState.Confirmation.FAILED -> {
Log.w(TAG, "3D signal was not confirmed at ${width}x$height; restoring 2D")
if (noSignal) markSbsFailed(device?.productId, true)
rememberSbs(GlassesState.SBS_FAILS)
}
else -> Unit
}
if (sbsByDisplay) {
markSbsFailed(device?.productId, false)
rememberSbs(GlassesState.SBS_WORKS)
}
if (controllable && device != null && !permitted && (ask || !asked) && _state.value.permission != Permission.ASKING) askPermission(device)
if (device != null && permitted && link == null) openLink(device)
publish {
it.copy(
present = device != null || display != null,
model = m?.name ?: display?.let(::displayName) ?: "VITURE",
controllable = controllable,
permission = when {
!controllable -> Permission.NONE
permitted -> Permission.GRANTED
it.permission == Permission.ASKING || it.permission == Permission.DENIED -> it.permission
else -> Permission.NONE
},
connected = link?.ready == true,
controlError = when {
m is VitureProtocol.Model && m.needsSdk && !VitureSdk.available -> app.getString(R.string.glasses_driver_missing)
device == null || link?.ready == true -> null
else -> it.controlError
},
displayId = display?.displayId ?: Display.INVALID_DISPLAY,
width = width, height = height, hz = mode?.refreshRate ?: 0f,
sbs = sbsByDisplay,
switching3d = modeState.pendingStereo,
brightnessMax = m?.brightnessMax ?: 8, volumeMax = m?.volumeMax ?: 8,
sunblock = if (m === XrealModel) it.sunblock else _settings.value.sunblock ?: it.sunblock,
hardwareControls = device?.vendorId == VitureProtocol.VENDOR_ID || m === XrealModel,
hasSunblock = m?.hasSunblock == true,
diagonalFovDeg = m?.diagonalFovDeg ?: 46f,
tracking = it.tracking && link != null,
noSignal = noSignal,
sbs90 = m?.sbs90 == true,
experimental = m === XrealModel,
)
}
if (link != null) applyMode(force = false)
scheduleModeCheck()
}
internal fun isGlassesVendor(vid: Int) = vid == VitureProtocol.VENDOR_ID || RayNeoModel.isVendor(vid) || vid == XrealModel.VENDOR_ID
private fun canOpenModel(m: GlassesModel): Boolean = m.controllable &&
(m !is VitureProtocol.Model || !m.needsSdk || VitureSdk.available)
internal fun deviceModel(device: UsbDevice): GlassesModel? {
val name = runCatching { device.productName }.getOrNull()
return if (device.vendorId == VitureProtocol.VENDOR_ID) {
VitureSdk.marketName(device.productId)?.let(VitureProtocol::sdkModel)
?: VitureProtocol.model(device.productId, name)
}
else if (XrealModel.matches(device.vendorId, device.productId)) XrealModel
else RayNeoModel.model(device.vendorId, device.productId, name)
}
internal fun isGlassesDisplayName(name: String) =
name.contains("VITURE", ignoreCase = true) || RayNeoModel.isDisplay(name) || XrealModel.isDisplay(name)
private fun findDisplay(usbPresent: Boolean): Display? {
return selectGlassesDisplay(displays.getDisplays(DisplayManager.DISPLAY_CATEGORY_PRESENTATION).toList(), usbPresent)
}
internal fun selectGlassesDisplay(displays: List<Display>, usbPresent: Boolean): Display? {
val candidates = displays
.filter { it.displayId != Display.DEFAULT_DISPLAY && it.flags and Display.FLAG_PRESENTATION != 0 }
// Android can expose the glasses name on either the logical display or
// its product metadata. A generic label on one must not hide the other.
return candidates.firstOrNull { display ->
isGlassesDisplayName(display.name) ||
(Build.VERSION.SDK_INT >= 31 && display.deviceProductInfo?.let { product ->
product.name?.let(::isGlassesDisplayName) == true ||
RayNeoModel.isDisplayProduct(product.name, product.manufacturerPnpId, product.productId)
} == true)
}
?: candidates.singleOrNull()?.takeIf { usbPresent && unnamedPanel(it) }
}
private fun unnamedPanel(d: Display): Boolean {
if (Build.VERSION.SDK_INT < 31) return true
return d.deviceProductInfo?.name.isNullOrBlank()
}
private fun displayName(d: Display): String? {
if (Build.VERSION.SDK_INT >= 31) d.deviceProductInfo?.name?.takeIf { it.isNotBlank() }?.let { return it }
return d.name
}
private fun askPermission(device: UsbDevice) {
asked = true
publish { it.copy(permission = Permission.ASKING) }
val flags = if (Build.VERSION.SDK_INT >= 31) PendingIntent.FLAG_MUTABLE else 0
val pi = PendingIntent.getBroadcast(app, 0, Intent(ACTION_PERMISSION).setPackage(app.packageName), flags)
runCatching { usb.requestPermission(device, pi) }.onFailure { Log.w(TAG, "requestPermission", it) }
}
private fun openLink(device: UsbDevice) {
val generation = ++linkGeneration
calibrationKey = calibrationStore.key(device.vendorId, device.productId,
runCatching { device.serialNumber }.getOrNull(), model?.name.orEmpty())
val l = if (device.vendorId == VitureProtocol.VENDOR_ID) {
if ((model as? VitureProtocol.Model)?.needsSdk == true) {
VitureSdkLink.open(usb, device, File(app.cacheDir, "viture"), sdkListener(generation),
(model as? VitureProtocol.Model)?.family)
} else VitureLink.open(usb, device, linkListener(generation))
} else if (model === XrealModel) {
XrealLink.open(app, xrealListener(generation))
} else {
val baseKey = calibrationKey!!
val gtUsb = RayNeoProtocol.isGt(device.vendorId, device.productId)
RayNeoLink.open(usb, device, rayNeoListener(generation)) { board ->
calibrationStore.load(RayNeoCalibrationStore.profileKey(baseKey, gtUsb, board))
}
}
val opened = l ?: run {
linkGeneration++
Log.w(TAG, "could not open ${device.deviceName}")
publish { it.copy(controlError = app.getString(R.string.glasses_connection_failed)) }
return
}
link = opened
_calibration.value = GlassesCalibrationState(supported = opened is RayNeoLink,
saved = opened is RayNeoLink && calibrationKey?.let(calibrationStore::load) != null)
linkDevice = device
glassesMode = 0
autoModeSent = false
Log.i(TAG, "opened ${model?.name} pid=0x%04X imu=%s".format(device.productId, opened.hasImu))
publish { it.copy(connected = opened.ready, controlError = null) }
listOf(VitureProtocol.MSG_GET_BRIGHTNESS, VitureProtocol.MSG_GET_VOLUME, VitureProtocol.MSG_GET_DISPLAY).forEachIndexed { i, msg ->
handler.postDelayed({ if (link === opened) opened.send(msg) }, VitureLink.COMMAND_GAP_MS * (i + 1))
}
handler.postDelayed({
if (link !== opened) return@postDelayed
_settings.value.sunblock?.let { sendFilm(it) }
updateImu()
applyMode(force = false)
}, 300)
}
private fun xrealListener(generation: Long) = object : XrealLink.Listener {
override fun onReady() = post {
if (generation != linkGeneration) return@post
publish { it.copy(connected = true, controlError = null) }
autoModeSent = false
updateImu()
applyMode(force = false)
}
override fun onValue(message: Int, value: Int) = post {
if (generation != linkGeneration) return@post
when (message) {
VitureProtocol.MSG_GET_DISPLAY -> glassesMode = value
VitureProtocol.MSG_BRIGHTNESS -> publish { it.copy(brightness = value) }
VitureProtocol.MSG_FILM -> publish { it.copy(sunblock = value != 0) }
}
}
override fun onPose(raw: ByteArray, pose: FloatArray, timeNs: Long) {
if (generation != linkGeneration) return
imuPackets++
probe?.invoke(raw, pose)
if (simulating.get() == null) deliver(pose, timeNs)
}
override fun onUnavailable(networkMissing: Boolean) = post {
if (generation != linkGeneration) return@post
publish { it.copy(connected = false, tracking = false,
controlError = app.getString(if (networkMissing) R.string.glasses_xreal_network_missing else R.string.glasses_xreal_connection_failed)) }
}
}
private fun sdkListener(generation: Long) = object : VitureSdkLink.Listener {
override fun onValue(msgId: Int, value: Int) = post {
if (generation != linkGeneration) return@post
when (msgId) {
VitureProtocol.MSG_GET_DISPLAY -> glassesMode = value
VitureProtocol.MSG_GET_BRIGHTNESS -> publish { it.copy(brightness = value) }
VitureProtocol.MSG_GET_VOLUME -> publish { it.copy(volume = value) }
VitureProtocol.MSG_FILM -> publish { it.copy(sunblock = value != 0) }
}
}
override fun onPose(q: FloatArray, timeNs: Long) {
if (generation != linkGeneration) return
imuPackets++
probe?.invoke(null, q)
if (simulating.get() == null) deliver(q, timeNs)
}
override fun onLost() = post {
if (generation != linkGeneration) return@post
closeLink()
publish { it.copy(controlError = app.getString(R.string.glasses_connection_failed)) }
// Let a reconnect or explicit Allow retry restart the SDK, not an endless failure loop.
}
}
private fun rayNeoListener(generation: Long) = object : RayNeoLink.Listener {
override fun onDeviceInfo(boardId: Int, calibrationSaved: Boolean) = post {
if (generation != linkGeneration) return@post
val device = linkDevice ?: return@post
val base = calibrationStore.key(device.vendorId, device.productId,
runCatching { device.serialNumber }.getOrNull(), model?.name.orEmpty())
calibrationKey = RayNeoCalibrationStore.profileKey(base,
RayNeoProtocol.isGt(device.vendorId, device.productId), boardId)
_calibration.value = _calibration.value.copy(saved = calibrationSaved)
}
override fun onCalibration(revision: Long, phase: CalibrationPhase, progress: Int) = post {
if (generation != linkGeneration || revision != calibrationRevision) return@post
if (phase != CalibrationPhase.IDLE && !_calibration.value.running) return@post
_calibration.value = _calibration.value.copy(phase = phase, progress = progress)
if (!_calibration.value.running) updateImu()
}
override fun onCalibrationSaved(revision: Long, value: RayNeoCalibration?) = post {
if (generation != linkGeneration || revision != calibrationRevision) return@post
if (value != null && !_calibration.value.running) return@post
calibrationKey?.let { calibrationStore.save(it, value) }
_calibration.value = _calibration.value.copy(saved = value != null)
}
override fun onReady() = post {
if (generation == linkGeneration) {
publish { it.copy(connected = true) }
updateImu()
applyMode(force = false)
}
}
override fun onPose(raw: ByteArray, xyzw: FloatArray, timeNs: Long) {
if (generation != linkGeneration) return
imuPackets++
probe?.invoke(raw, xyzw)
if (simulating.get() == null) deliver(xyzw, timeNs)
}
override fun onLost() = post {
if (generation != linkGeneration) return@post
closeLink()
refresh()
}
}
private fun closeLink() {
val l = link ?: return
linkGeneration++
link = null
linkDevice = null
calibrationKey = null
_calibration.value = GlassesCalibrationState(phase = if (_calibration.value.running) CalibrationPhase.FAILED else CalibrationPhase.IDLE)
glassesMode = 0
l.close()
publish { it.copy(connected = false, tracking = false) }
Log.i(TAG, "link closed")
}
private fun rememberSbs(support: Int) {
if (_state.value.sbsSupport == support) return
publish { it.copy(sbsSupport = support) }
}
private fun sbsFailed(productId: Int): Boolean =
app.getSharedPreferences(PREFS, Context.MODE_PRIVATE).getStringSet(KEY_SBS_FAILED, null)?.contains(productId.toString(16)) == true
private fun markSbsFailed(productId: Int?, failed: Boolean) {
if (productId == null || sbsFailed(productId) == failed) return
val prefs = app.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
val ids = prefs.getStringSet(KEY_SBS_FAILED, null).orEmpty()
val id = productId.toString(16)
prefs.edit().putStringSet(KEY_SBS_FAILED, if (failed) ids + id else ids - id).apply()
}
private fun applyMode(force: Boolean) {
val s = _state.value
val m = model ?: return
if (!s.canControl) return
if (modeState.restorePending) {
sendMode(false)
return
}
if (!s.usable || s.sbs || modeState.pendingStereo) return
val hz = _settings.value.refreshHz
if (glassesMode == m.mode(hz, false)) return
if (!force) {
if (autoModeSent || SystemClock.uptimeMillis() - lastModeSentAt < 8_000) return
if (glassesMode == 0 && abs(s.hz - hz) < 1.5f) return
autoModeSent = true
}
sendMode(false)
}
private fun sendMode(sbs: Boolean): Boolean {
val m = model ?: return false
val g = _settings.value
val mode = m.mode(if (sbs) g.sbsHz else g.refreshHz, sbs)
if (link?.send(VitureProtocol.MSG_DISPLAY, byteArrayOf(mode.toByte())) == true) {
if (link is RayNeoLink) glassesMode = mode
lastModeSentAt = SystemClock.uptimeMillis()
modeState.sent(sbs, lastModeSentAt)
publish { it.copy(switching3d = modeState.pendingStereo) }
scheduleModeCheck()
return true
}
return false
}
private fun scheduleModeCheck() {
handler.removeCallbacks(confirmMode)
modeState.nextCheckAt?.let { at ->
handler.postDelayed(confirmMode, (at - SystemClock.uptimeMillis()).coerceAtLeast(0))
}
}
private fun sendFilm(on: Boolean) {
if (!_state.value.canControl || !_state.value.hardwareControls || !_state.value.hasSunblock) return
if (link?.send(VitureProtocol.MSG_FILM, byteArrayOf(if (on) 1 else 0)) == true && link !is XrealLink)
publish { it.copy(sunblock = on) }
}
private fun updateImu() {
val l = link ?: return
if (!l.ready) return
val want = (sink != null || probe != null || _calibration.value.running) && l.hasImu
if (want == _state.value.tracking) return
val accepted = l.send(VitureProtocol.MSG_IMU, byteArrayOf(if (want) 1 else 0))
if (want && !accepted) return
if (want) l.send(VitureProtocol.MSG_IMU_RATE, byteArrayOf(VitureProtocol.IMU_RATE_240.toByte()))
publish { it.copy(tracking = want) }
}
private fun linkListener(generation: Long) = object : VitureLink.Listener {
override fun onFrame(frame: VitureProtocol.Frame) = post {
if (generation != linkGeneration) return@post
val line = "%s 0x%04X %s".format(
when (frame.magic) { VitureProtocol.MAGIC_REPORT -> "FD"; VitureProtocol.MAGIC_COMMAND -> "FE"; else -> "FC" },
frame.msgId, frame.payload.joinToString("") { "%02X".format(it) },
)
synchronized(frames) {
frames.addLast(line)
while (frames.size > 64) frames.removeFirst()
}
Log.i(TAG, "frame $line")
onReport(frame)
}
override fun onImu(payload: ByteArray, timeNs: Long) {
if (generation != linkGeneration) return
imuPackets++
probe?.invoke(payload, null)
if (simulating.get() != null) return
VitureProtocol.imuQuaternion(payload)?.let { deliver(it, timeNs) }
}
override fun onLost() = post {
if (generation != linkGeneration) return@post
closeLink()
refresh()
}
}
private fun deliver(q: FloatArray, timeNs: Long) {
latest = q
sink?.onPose(q[0], q[1], q[2], q[3], timeNs)
}
private fun onReport(frame: VitureProtocol.Frame) {
val p = frame.payload
val event = p.firstOrNull()?.toInt()?.and(0xFF) ?: return
val answer = if (p.size >= 2 && event == 0) p[1].toInt() and 0xFF else null
when (frame.msgId) {
VitureProtocol.EVENT_DISPLAY -> {
Log.i(TAG, "glasses switched to mode 0x%02X".format(event))
glassesMode = event
}
VitureProtocol.EVENT_DISPLAY_STEP -> model?.modeForStep(event)?.let { mode ->
Log.i(TAG, "glasses report mode 0x%02X".format(mode))
glassesMode = mode
}
VitureProtocol.EVENT_BRIGHTNESS -> publish { it.copy(brightness = event) }
VitureProtocol.EVENT_VOLUME -> publish { it.copy(volume = event) }
VitureProtocol.MSG_GET_BRIGHTNESS, VitureProtocol.MSG_BRIGHTNESS -> answer?.let { v -> publish { it.copy(brightness = v) } }
VitureProtocol.MSG_GET_VOLUME, VitureProtocol.MSG_VOLUME -> answer?.let { v -> publish { it.copy(volume = v) } }
VitureProtocol.MSG_DISPLAY -> if (event != 0) {
Log.w(TAG, "glasses refused display mode 0x%02X".format(p.getOrNull(1)?.toInt()?.and(0xFF) ?: 0))
modeState.cancelPending()
publish { it.copy(switching3d = false) }
scheduleModeCheck()
} else answer?.let { glassesMode = it }
VitureProtocol.MSG_GET_DISPLAY -> answer?.let { v ->
glassesMode = v
}
}
}
fun sampleImu(ms: Long): JSONObject {
val raw = java.util.Collections.synchronizedList(mutableListOf<Pair<ByteArray?, FloatArray?>>())
val counter = java.util.concurrent.atomic.AtomicInteger()
val began = SystemClock.elapsedRealtime()
probe = { payload, pose -> val n = counter.incrementAndGet(); if (raw.size < 4 || n % 60 == 0) raw.add(payload?.copyOf() to pose?.copyOf()) }
post { updateImu() }
SystemClock.sleep(ms)
probe = null
post { updateImu() }
val seconds = (SystemClock.elapsedRealtime() - began) / 1000.0
val count = counter.get()
val samples = JSONArray()
synchronized(raw) {
raw.forEach { (b, pose) ->
val rayneo = b != null && RayNeoProtocol.classifyPacket(b) is RayNeoPacket.Sensor
val q = pose ?: if (b == null || rayneo) null else VitureProtocol.imuQuaternion(b)
samples.put(JSONObject()
.put("hex", b?.joinToString("") { "%02X".format(it) } ?: JSONObject.NULL)
.put("euler", if (b == null || b.size < 12 || rayneo || pose != null) JSONObject.NULL else JSONArray().put(VitureProtocol.beFloat(b, 0).toDouble()).put(VitureProtocol.beFloat(b, 4).toDouble()).put(VitureProtocol.beFloat(b, 8).toDouble()))
.put("quat", q?.let { JSONArray().put(it[0].toDouble()).put(it[1].toDouble()).put(it[2].toDouble()).put(it[3].toDouble()) } ?: JSONObject.NULL))
}
}
return JSONObject().put("packets", count).put("rateHz", count / seconds).put("samples", samples)
}
private fun Intent.device(): UsbDevice? =
if (Build.VERSION.SDK_INT >= 33) getParcelableExtra(UsbManager.EXTRA_DEVICE, UsbDevice::class.java)
else @Suppress("DEPRECATION") getParcelableExtra(UsbManager.EXTRA_DEVICE)
private fun load(): GlassesSettings {
val p = app.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
return GlassesSettings(
refreshHz = p.getInt("refreshHz", 120),
sbsHz = p.getInt("sbsHz", 60),
sunblock = if (p.contains("sunblock")) p.getBoolean("sunblock", false) else null,
gameOnGlasses = p.getBoolean("gameOnGlasses", true),
headTracking = p.getBoolean("headTracking", true),
auto3d = p.getBoolean("auto3d", GlassesSettings().auto3d),
picture3d = p.getInt("picture3d", GlassesSettings.PICTURE_AUTO),
screenWidthDm = p.getInt("screenWidthDm", 20),
screenDistanceDm = p.getInt("screenDistanceDm", 30),
vrFovDeg = p.getInt("vrFovDeg", GlassesSettings.VR_FOV_DEFAULT).coerceIn(GlassesSettings.VR_FOV_MIN, GlassesSettings.VR_FOV_MAX),
)
}
private fun save(s: GlassesSettings) {
app.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit().apply {
putInt("refreshHz", s.refreshHz)
putInt("sbsHz", s.sbsHz)
s.sunblock?.let { putBoolean("sunblock", it) } ?: remove("sunblock")
putBoolean("gameOnGlasses", s.gameOnGlasses)
putBoolean("headTracking", s.headTracking)
putBoolean("auto3d", s.auto3d)
putInt("picture3d", s.picture3d)
putInt("screenWidthDm", s.screenWidthDm)
putInt("screenDistanceDm", s.screenDistanceDm)
putInt("vrFovDeg", s.vrFovDeg)
}.apply()
}
}
@@ -0,0 +1,25 @@
package com.droiddeck.launcher.glasses
/** Hardware properties; display refresh is selected from Android's advertised modes. */
interface GlassesModel {
val name: String
val controllable: Boolean
val brightnessMax: Int
val volumeMax: Int
val diagonalFovDeg: Float
val sbs90: Boolean
val hasSunblock: Boolean get() = true
fun mode(hz: Int, sbs: Boolean): Int
fun modeForStep(step: Int): Int?
}
/** The command IDs here are the existing glasses controller's internal vocabulary.
* Each transport translates them to its own wire protocol; unsupported controls return false.
*/
internal interface GlassesLink {
val hasImu: Boolean
val badFrames: Long get() = 0L
val ready: Boolean get() = true
fun send(msgId: Int, payload: ByteArray = ByteArray(0)): Boolean
fun close()
}
@@ -0,0 +1,84 @@
package com.droiddeck.launcher.glasses
internal class GlassesModeState(private val timeoutMs: Long) {
enum class Confirmation { NONE, WAITING, CONFIRMED, FAILED }
data class Device(val productId: Int, val serial: String?)
private var deadline: Long? = null
private var device: Device? = null
private var attached = false
private var reconnectDeadline: Long? = null
private var vrOwned = false
var restorePending = false
private set
val pendingStereo: Boolean get() = deadline != null
val nextCheckAt: Long? get() = listOfNotNull(deadline, reconnectDeadline).minOrNull()
fun updateDevice(current: Device?, now: Long): Boolean {
if (reconnectDeadline?.let { now >= it } == true) abandon()
if (current == null) {
if (attached) reconnectDeadline = now + timeoutMs
attached = false
return false
}
val previous = device
val changed = previous == null || previous.productId != current.productId ||
(previous.serial != null && current.serial != null && previous.serial != current.serial)
if (changed) abandon()
device = if (!changed && current.serial == null) previous else current
attached = true
reconnectDeadline = null
return changed
}
private fun abandon() {
deadline = null
reconnectDeadline = null
vrOwned = false
restorePending = false
}
fun manual() {
vrOwned = false
restorePending = false
}
fun startVr() {
vrOwned = true
restorePending = false
}
fun stopVr(): Boolean {
if (vrOwned) {
vrOwned = false
restorePending = true
}
return restorePending
}
fun sent(stereo: Boolean, now: Long) {
deadline = if (stereo) now + timeoutMs else null
if (!stereo) restorePending = false
}
fun cancelPending() {
deadline = null
}
fun observe(stereo: Boolean, now: Long): Confirmation {
val until = deadline ?: return Confirmation.NONE
if (stereo) {
deadline = null
return Confirmation.CONFIRMED
}
if (now < until) return Confirmation.WAITING
if (!attached) {
abandon()
return Confirmation.NONE
}
deadline = null
vrOwned = false
restorePending = true
return Confirmation.FAILED
}
}
@@ -0,0 +1,67 @@
package com.droiddeck.launcher.glasses
import com.droiddeck.launcher.gpu.Cardboard
/** Share the actual picture mode with settings so controls match the rendered view. */
internal object GlassesRendering {
enum class Picture { FLAT, VIRTUAL_SCREEN, STEREO }
fun outputHz(settings: GlassesSettings, state: GlassesState): Int =
if (state.sbs) if (state.sbs90) settings.sbsHz else 60 else settings.refreshHz
fun mirrorRefreshLimit(settings: GlassesSettings, state: GlassesState, outputActive: Boolean): Int =
if (state.usable && !outputActive) outputHz(settings, state) else 0
fun fpsLimit(requested: Int, mirrorRefreshLimit: Int): Int = when {
mirrorRefreshLimit <= 0 -> requested
requested <= 0 -> mirrorRefreshLimit
else -> minOf(requested, mirrorRefreshLimit)
}
fun usesGlasses(state: GlassesState, outputActive: Boolean, steamVrRunning: Boolean): Boolean =
outputActive || (steamVrRunning && state.usable)
fun picture(settings: GlassesSettings, stereoDisplay: Boolean, steamVrRunning: Boolean): Picture =
when (settings.picture3d) {
GlassesSettings.PICTURE_SCREEN -> Picture.VIRTUAL_SCREEN
GlassesSettings.PICTURE_SBS -> Picture.STEREO
else -> when {
steamVrRunning -> Picture.STEREO
stereoDisplay -> Picture.VIRTUAL_SCREEN
else -> Picture.FLAT
}
}
fun cardboard(
base: Cardboard,
settings: GlassesSettings,
state: GlassesState,
outputActive: Boolean,
steamVrRunning: Boolean,
): Cardboard {
// Mirror output still uses the glasses' picture and sensors during VR.
if (!usesGlasses(state, outputActive, steamVrRunning)) {
// SteamVR uses the renderer's enabled flag as headset proximity.
// A flat phone preview still needs an active HMD and one SBS eye;
// otherwise SteamVR enters standby and leaves a frozen frame.
return if (steamVrRunning && !base.enabled) base.copy(
enabled = true,
source = Cardboard.SOURCE_SBS,
viewer = Cardboard.Viewer.NONE.id,
mono = true,
) else base
}
val picture = picture(settings, state.sbs, steamVrRunning)
if (picture == Picture.FLAT) return Cardboard.OFF
return Cardboard(
enabled = true,
source = if (picture == Picture.STEREO) Cardboard.SOURCE_SBS else Cardboard.SOURCE_SCREEN,
viewer = Cardboard.Viewer.VITURE.id,
ipdMm = base.ipdMm,
screenWidthDm = settings.screenWidthDm,
screenDistanceDm = settings.screenDistanceDm,
headTracking = settings.headTracking && state.canControl,
mono = !state.sbs,
)
}
}
@@ -0,0 +1,51 @@
package com.droiddeck.launcher.glasses
import android.app.Presentation
import android.content.Context
import android.graphics.Color
import android.os.Bundle
import android.util.Log
import android.view.Display
import android.view.SurfaceHolder
import android.view.SurfaceView
import android.view.ViewGroup
import android.view.WindowManager
import android.widget.FrameLayout
import kotlin.math.abs
class GlassesScreen(context: Context, display: Display, refreshHz: Int, private val callback: SurfaceHolder.Callback) : Presentation(context, display) {
val surfaceView = SurfaceView(getContext())
val holder: SurfaceHolder get() = surfaceView.holder
private var wantHz = refreshHz
val hz: Float get() = display.mode.refreshRate
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
val root = FrameLayout(context).apply { setBackgroundColor(Color.BLACK) }
root.addView(surfaceView, FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT))
setContentView(root)
window?.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
applyMode()
surfaceView.holder.addCallback(callback)
}
fun prefer(hz: Int) {
wantHz = hz
// A 2D/SBS transition can replace the active mode without changing Hz.
applyMode()
}
private fun applyMode() {
val w = window ?: return
val current = display.mode
val best = display.supportedModes.filter {
it.physicalWidth == current.physicalWidth && it.physicalHeight == current.physicalHeight
}.minByOrNull { abs(it.refreshRate - wantHz) } ?: return
if (w.attributes.preferredDisplayModeId != best.modeId) {
Log.i("GlassesScreen", "display ${display.displayId}: prefer mode ${best.modeId} " +
"${best.physicalWidth}x${best.physicalHeight}@${best.refreshRate} (requested $wantHz Hz)")
w.attributes = w.attributes.apply { preferredDisplayModeId = best.modeId }
}
}
}
@@ -0,0 +1,105 @@
package com.droiddeck.launcher.glasses
import android.content.SharedPreferences
import org.json.JSONArray
import org.json.JSONObject
enum class CalibrationPhase { IDLE, SETTLING, COLLECTING, MOVING, TURN_OVER, SAVED, FAILED }
data class GlassesCalibrationState(
val supported: Boolean = false,
val saved: Boolean = false,
val phase: CalibrationPhase = CalibrationPhase.IDLE,
val progress: Int = 0,
) {
val running: Boolean get() = phase in setOf(CalibrationPhase.SETTLING, CalibrationPhase.COLLECTING,
CalibrationPhase.MOVING, CalibrationPhase.TURN_OVER)
}
internal data class RayNeoCalibration(val gyroBiasDps: Vec3, val level: Quaternion) {
fun encode(): String = JSONObject().put("version", 1)
.put("bias", JSONArray(listOf(gyroBiasDps.x, gyroBiasDps.y, gyroBiasDps.z)))
.put("level", JSONArray(listOf(level.w, level.x, level.y, level.z))).toString()
companion object {
fun decode(json: String?): RayNeoCalibration? = runCatching {
val obj = JSONObject(json ?: return null)
require(obj.getInt("version") == 1)
val b = obj.getJSONArray("bias"); val q = obj.getJSONArray("level")
require(b.length() == 3 && q.length() == 4)
val bias = Vec3(b.getDouble(0).toFloat(), b.getDouble(1).toFloat(), b.getDouble(2).toFloat())
val level = Quaternion(q.getDouble(0).toFloat(), q.getDouble(1).toFloat(), q.getDouble(2).toFloat(), q.getDouble(3).toFloat())
require(listOf(bias.x, bias.y, bias.z, level.w, level.x, level.y, level.z).all { it.isFinite() })
require(bias.length() <= 6f && level.w in 0.96f..1.001f)
val norm = level.w * level.w + level.x * level.x + level.y * level.y + level.z * level.z
require(norm in 0.99f..1.01f)
RayNeoCalibration(bias, level.normalized())
}.getOrNull()
}
}
internal class RayNeoCalibrationStore(private val prefs: SharedPreferences) {
companion object {
// GT and GT Max can share USB identity fields. Keep the non-Max mounting
// correction separate from existing Max profiles and their legacy keys.
fun profileKey(base: String, gtUsb: Boolean, boardId: Int): String =
if (gtUsb && boardId == RayNeoProtocol.BOARD_GT) "$base:gt-standard-v1" else base
}
// A serial isolates physical units. Devices without one share their model profile.
fun key(vid: Int, pid: Int, serial: String?, modelName: String = ""): String =
"rayneo-cal-v1:$vid:$pid:${serial?.takeIf { it.isNotBlank() } ?: "model:$modelName"}"
fun load(key: String): RayNeoCalibration? = RayNeoCalibration.decode(prefs.getString(key, null))
fun save(key: String, value: RayNeoCalibration?) {
if (value == null) prefs.edit().remove(key).apply()
else prefs.edit().putString(key, value.encode()).apply()
}
}
/** One upside-down stationary pose estimates gyro bias and tilt, never yaw or accelerometer scale. */
internal class RayNeoCalibrationCollector(private val axes: RayNeoSensorAxes, private val beganMs: Long) {
var phase = CalibrationPhase.SETTLING; private set
var progress = 0; private set
private var windowStart = -1L
private var lastMs = -1L
private var count = 0
private var meanA = Vec3(0f, 0f, 0f)
private var meanG = Vec3(0f, 0f, 0f)
private var aVariance = 0f
private var gVariance = 0f
private fun clear(next: CalibrationPhase) {
phase = next; progress = 0; windowStart = -1; count = 0
meanA = Vec3(0f, 0f, 0f); meanG = meanA; aVariance = 0f; gVariance = 0f
}
fun update(sample: RayNeoSensorSample, nowMs: Long): RayNeoCalibration? {
if (phase == CalibrationPhase.FAILED || phase == CalibrationPhase.SAVED) return null
if (nowMs - beganMs >= 30_000) { clear(CalibrationPhase.FAILED); return null }
val gap = lastMs >= 0 && nowMs - lastMs > 250
lastMs = nowMs
if (nowMs - beganMs < 3000) return null
val a = axes.map(Vec3(sample.accelMps2[0], sample.accelMps2[1], sample.accelMps2[2]))
val g = Vec3(sample.gyroDps[0], sample.gyroDps[1], sample.gyroDps[2])
if (a.length() !in 8.5f..11f || !g.length().isFinite() || g.length() > 6f || gap) {
clear(CalibrationPhase.MOVING); return null
}
if (a.normalized().y > -0.94f) { clear(CalibrationPhase.TURN_OVER); return null }
if (windowStart < 0) windowStart = nowMs
phase = CalibrationPhase.COLLECTING
count++
val da = a - meanA; val dg = g - meanG
meanA += da * (1f / count); meanG += dg * (1f / count)
fun dot(x: Vec3, y: Vec3) = x.x * y.x + x.y * y.y + x.z * y.z
aVariance += dot(da, a - meanA); gVariance += dot(dg, g - meanG)
if (count > 25 && (aVariance / count > 0.04f || gVariance / count > 2.25f)) {
clear(CalibrationPhase.MOVING); return null
}
progress = ((nowMs - windowStart) * 100 / 5000).toInt().coerceIn(0, 99)
if (nowMs - windowStart < 5000 || count < 200) return null
val down = meanA.normalized()
val axis = down.cross(Vec3(0f, -1f, 0f))
val result = RayNeoCalibration(meanG, Quaternion(1f - down.y, axis.x, axis.y, axis.z).normalized())
phase = CalibrationPhase.SAVED; progress = 100
return result
}
}
@@ -0,0 +1,202 @@
package com.droiddeck.launcher.glasses
import android.hardware.usb.UsbDevice
import android.hardware.usb.UsbDeviceConnection
import android.hardware.usb.UsbManager
import android.os.SystemClock
import android.util.Log
/** RayNeo command/response and IMU share one endpoint. Keep reads on a single worker. */
internal class RayNeoLink private constructor(
private val connection: UsbDeviceConnection,
private val endpoints: RayNeoEndpointSelection,
private val listener: Listener,
private val gt: Boolean,
private val loadCalibration: (Int) -> RayNeoCalibration?,
) : GlassesLink {
interface Listener {
fun onReady()
fun onPose(raw: ByteArray, xyzw: FloatArray, timeNs: Long)
fun onLost()
fun onCalibration(revision: Long, phase: CalibrationPhase, progress: Int) {}
fun onCalibrationSaved(revision: Long, value: RayNeoCalibration?) {}
fun onDeviceInfo(boardId: Int, calibrationSaved: Boolean) {}
}
@Volatile private var running = true
@Volatile override var ready = false
private set
@Volatile private var wantImu = false
private val writeLock = Any()
private val assembler = RayNeoPacketAssembler()
private val readBuffer = ByteArray(maxOf(endpoints.inEndpoint.maxPacketSize, 64))
private var failures = 0
private lateinit var worker: Thread
private var calibration: RayNeoCalibration? = null
private data class CalibrationCommand(val revision: Long, val action: Int, val saved: RayNeoCalibration? = null)
private val calibrationCommand = java.util.concurrent.atomic.AtomicReference<CalibrationCommand?>(null)
fun startCalibration(revision: Long) { calibrationCommand.set(CalibrationCommand(revision, 1)) }
fun cancelCalibration(revision: Long, saved: RayNeoCalibration?) { calibrationCommand.set(CalibrationCommand(revision, 2, saved)) }
fun resetCalibration(revision: Long) { calibrationCommand.set(CalibrationCommand(revision, 3)) }
override val hasImu = true
override fun send(msgId: Int, payload: ByteArray): Boolean {
if (!running) return false
return when (msgId) {
VitureProtocol.MSG_IMU -> {
wantImu = payload.firstOrNull() == 1.toByte()
true
}
VitureProtocol.MSG_DISPLAY -> {
val mode = payload.firstOrNull()?.toInt()?.and(0xff)
ready && (mode == RayNeoProtocol.CMD_SWITCH_TO_3D || mode == RayNeoProtocol.CMD_SWITCH_TO_2D) &&
command(mode)
}
// No VITURE brightness, volume, film, rate or raw command is sent to RayNeo.
else -> false
}
}
private fun command(cmd: Int): Boolean = synchronized(writeLock) {
if (!running) return false
val packet = RayNeoProtocol.buildCommandPacket(cmd, 0, endpoints.outEndpoint.maxPacketSize)
connection.bulkTransfer(endpoints.outEndpoint, packet, packet.size, 250) == packet.size
}
private fun read(timeoutMs: Int): ByteArray? {
val deadline = SystemClock.elapsedRealtime() + timeoutMs
while (running && SystemClock.elapsedRealtime() < deadline) {
assembler.nextPacket()?.let { return it }
val began = SystemClock.elapsedRealtime()
val count = connection.bulkTransfer(endpoints.inEndpoint, readBuffer, readBuffer.size, 50)
if (count > 0) {
failures = 0
assembler.append(readBuffer, count)
} else if (count < 0 && SystemClock.elapsedRealtime() - began < 25) {
if (++failures >= 50) error("RayNeo USB link lost")
SystemClock.sleep(5)
}
}
return null
}
private fun response(cmd: Int, timeoutMs: Int): RayNeoPacket.Response? {
val deadline = SystemClock.elapsedRealtime() + timeoutMs
while (running && SystemClock.elapsedRealtime() < deadline) {
val packet = read(100)?.let(RayNeoProtocol::classifyPacket)
if (packet is RayNeoPacket.Response && packet.cmd == cmd) return packet
}
return null
}
private fun run() {
var streaming = false
try {
check(command(RayNeoProtocol.CMD_ACQUIRE_DEVICE_INFO)) { "RayNeo device-info write failed" }
val info = response(RayNeoProtocol.CMD_ACQUIRE_DEVICE_INFO, 2500)
?.let { RayNeoProtocol.parseDeviceInfo(it.raw) } ?: error("RayNeo device-info timed out")
val profile = RayNeoSensorProfile.forDevice(gt, info.boardId)
val rotation = profile.rotationXDeg
val gtMaxAxes = profile.gtMaxAxes
calibration = loadCalibration(info.boardId)
listener.onDeviceInfo(info.boardId, calibration != null)
Log.i("RayNeoLink", "Device board=0x%02x gt=%s".format(info.boardId, gt))
var filter = RayNeoOrientationFilter(rotation, gtMaxAxes, calibration)
var collector: RayNeoCalibrationCollector? = null
var calibrationRevision = 0L
var lastCalibrationUpdate = 0L
var warmupUntil = 0L
var lastSensor = 0L
ready = true
listener.onReady()
while (running) {
val request = calibrationCommand.getAndSet(null)
if (request != null) calibrationRevision = request.revision
when (request?.action) {
1 -> collector = RayNeoCalibrationCollector(RayNeoSensorAxes(rotation, gtMaxAxes), SystemClock.elapsedRealtime())
2 -> {
collector = null
if (calibration != request.saved) {
calibration = request.saved
filter = RayNeoOrientationFilter(rotation, gtMaxAxes, calibration)
}
listener.onCalibration(calibrationRevision, CalibrationPhase.IDLE, 0)
}
3 -> {
collector = null; calibration = null
filter = RayNeoOrientationFilter(rotation, gtMaxAxes)
listener.onCalibrationSaved(calibrationRevision, null)
listener.onCalibration(calibrationRevision, CalibrationPhase.IDLE, 0)
}
}
if (wantImu != streaming) {
val cmd = if (wantImu) RayNeoProtocol.CMD_OPEN_IMU else RayNeoProtocol.CMD_CLOSE_IMU
check(command(cmd) && response(cmd, 1500) != null) { "RayNeo IMU command timed out" }
streaming = cmd == RayNeoProtocol.CMD_OPEN_IMU
filter = RayNeoOrientationFilter(rotation, gtMaxAxes, calibration)
lastSensor = SystemClock.elapsedRealtime()
warmupUntil = lastSensor + 2000
}
val raw = read(100)
if (streaming && SystemClock.elapsedRealtime() - lastSensor > 4000) error("RayNeo IMU stream stalled")
if (raw == null) continue
val packet = RayNeoProtocol.classifyPacket(raw)
if (streaming && packet is RayNeoPacket.Sensor) {
lastSensor = SystemClock.elapsedRealtime()
collector?.let { c ->
val result = c.update(packet.sample, lastSensor)
if (result != null && calibrationCommand.get() == null) {
calibration = result
filter = RayNeoOrientationFilter(rotation, gtMaxAxes, result)
warmupUntil = lastSensor + 2000
listener.onCalibrationSaved(calibrationRevision, result)
}
if (lastSensor - lastCalibrationUpdate >= 200 || result != null || c.phase == CalibrationPhase.FAILED) {
listener.onCalibration(calibrationRevision, c.phase, c.progress)
lastCalibrationUpdate = lastSensor
}
if (result != null || c.phase == CalibrationPhase.FAILED) collector = null
}
val wxyz = filter.update(packet.sample)
if (wantImu && collector == null && lastSensor >= warmupUntil) {
// Normalize basis and component order at the hardware boundary.
listener.onPose(raw, RayNeoOrientationFilter.glassesPose(wxyz),
SystemClock.elapsedRealtimeNanos())
}
}
}
} catch (e: Exception) {
if (running) Log.w("RayNeoLink", "RayNeo stream failed", e)
} finally {
ready = false
if (running) listener.onLost()
}
}
override fun close() {
synchronized(writeLock) {
if (!running) return
// Best effort: stop the sensor before releasing the claimed interface.
if (ready) runCatching { command(RayNeoProtocol.CMD_CLOSE_IMU) }
running = false
}
if (Thread.currentThread() !== worker) worker.join(500)
runCatching { connection.releaseInterface(endpoints.usbInterface) }
connection.close()
}
companion object {
fun open(usb: UsbManager, device: UsbDevice, listener: Listener, loadCalibration: (Int) -> RayNeoCalibration? = { null }): RayNeoLink? {
val endpoints = RayNeoProtocol.selectEndpoints(device) ?: return null
val conn = usb.openDevice(device) ?: return null
if (!conn.claimInterface(endpoints.usbInterface, true)) {
conn.close()
return null
}
return RayNeoLink(conn, endpoints, listener, RayNeoProtocol.isGt(device.vendorId, device.productId), loadCalibration).also { link ->
link.worker = Thread({ link.run() }, "rayneo-imu").apply { isDaemon = true }
link.worker.start()
}
}
}
}
@@ -0,0 +1,42 @@
package com.droiddeck.launcher.glasses
object RayNeoModel {
private val modelName = Regex("(?i)(ray\\s*neo|tcl).*\\b(air|gt)(?=\\b|[0-9])")
fun isVendor(vid: Int) = vid == RayNeoProtocol.VENDOR_ID ||
vid == RayNeoProtocol.GT_VENDOR_ID || vid == 0x22d9
fun model(vid: Int, pid: Int, name: String?): GlassesModel? {
if (!isVendor(vid)) return null
// GT's identity is confirmed by verncat's SDK; do not claim other products
// from that vendor just because their name resembles a supported model.
val gt = RayNeoProtocol.isGt(vid, pid)
if (vid == RayNeoProtocol.GT_VENDOR_ID && !gt) return null
if (!(vid == RayNeoProtocol.VENDOR_ID && pid == RayNeoProtocol.PRODUCT_ID) &&
!gt &&
!modelName.containsMatchIn(name.orEmpty())) return null
return Model(name?.trim()?.takeIf { modelName.containsMatchIn(it) }
?: if (gt) "RayNeo GT series" else "RayNeo Air")
}
fun isDisplay(name: String): Boolean =
name.contains("rayneo", ignoreCase = true) || name.contains("ray neo", ignoreCase = true) ||
modelName.containsMatchIn(name)
// Observed on GT Max: Android exposes an unbranded EDID monitor name.
// Match its product metadata, not every display called SmartGlasses/TCL.
fun isDisplayProduct(name: String?, manufacturer: String?, productId: String?): Boolean =
name?.trim().equals("SmartGlasses", ignoreCase = true) &&
manufacturer.equals("TCL", ignoreCase = true) && productId == "980"
private class Model(override val name: String) : GlassesModel {
override val controllable = true
override val brightnessMax = 0
override val volumeMax = 0
override val diagonalFovDeg = 46f // Conservative existing default, not a calibrated model profile.
override val sbs90 = false
override fun mode(hz: Int, sbs: Boolean) =
if (sbs) RayNeoProtocol.CMD_SWITCH_TO_3D else RayNeoProtocol.CMD_SWITCH_TO_2D
override fun modeForStep(step: Int): Int? = null
}
}
@@ -0,0 +1,190 @@
// Adapted from informalTechCode/RayNeo-Air-Series-3DoF-Scaffold (Apache-2.0).
// See docs/third-party/rayneo/. Returns scaffold quaternions in (w, x, y, z) order.
package com.droiddeck.launcher.glasses
import kotlin.math.cos
import kotlin.math.sin
import kotlin.math.sqrt
internal data class Vec3(val x: Float, val y: Float, val z: Float) {
fun length(): Float = sqrt(x * x + y * y + z * z)
fun normalized(): Vec3 {
val n = length()
if (n < 1e-6f) {
return Vec3(0f, 0f, 0f)
}
return Vec3(x / n, y / n, z / n)
}
fun cross(other: Vec3): Vec3 {
return Vec3(
y * other.z - z * other.y,
z * other.x - x * other.z,
x * other.y - y * other.x
)
}
operator fun plus(other: Vec3): Vec3 = Vec3(x + other.x, y + other.y, z + other.z)
operator fun minus(other: Vec3): Vec3 = Vec3(x - other.x, y - other.y, z - other.z)
operator fun times(scale: Float): Vec3 = Vec3(x * scale, y * scale, z * scale)
}
internal data class Quaternion(val w: Float, val x: Float, val y: Float, val z: Float) {
operator fun times(other: Quaternion): Quaternion {
return Quaternion(
w = w * other.w - x * other.x - y * other.y - z * other.z,
x = w * other.x + x * other.w + y * other.z - z * other.y,
y = w * other.y - x * other.z + y * other.w + z * other.x,
z = w * other.z + x * other.y - y * other.x + z * other.w
)
}
operator fun times(scale: Float): Quaternion {
return Quaternion(w * scale, x * scale, y * scale, z * scale)
}
fun normalized(): Quaternion {
val n = sqrt(w * w + x * x + y * y + z * z)
if (n < 1e-6f) {
return IDENTITY
}
return Quaternion(w / n, x / n, y / n, z / n)
}
fun conjugate(): Quaternion = Quaternion(w, -x, -y, -z)
fun rotate(v: Vec3): Vec3 {
val qv = Quaternion(0f, v.x, v.y, v.z)
val r = this * qv * conjugate()
return Vec3(r.x, r.y, r.z)
}
fun integrateBodyRate(omegaRadPerSec: Vec3, dtSec: Float): Quaternion {
val omega = Quaternion(0f, omegaRadPerSec.x, omegaRadPerSec.y, omegaRadPerSec.z)
val qDot = (this * omega) * 0.5f
return Quaternion(
w + qDot.w * dtSec,
x + qDot.x * dtSec,
y + qDot.y * dtSec,
z + qDot.z * dtSec
)
}
companion object {
val IDENTITY = Quaternion(1f, 0f, 0f, 0f)
fun fromAxisAngle(axis: Vec3, angleRad: Float): Quaternion {
val nAxis = axis.normalized()
val half = angleRad * 0.5f
val s = sin(half)
val c = cos(half)
return Quaternion(c, nAxis.x * s, nAxis.y * s, nAxis.z * s).normalized()
}
}
}
internal data class RayNeoSensorProfile(val rotationXDeg: Float, val gtMaxAxes: Boolean) {
companion object {
fun forDevice(gtUsb: Boolean, boardId: Int) = when {
gtUsb && boardId == RayNeoProtocol.BOARD_GT -> RayNeoSensorProfile(-25f, false)
gtUsb -> RayNeoSensorProfile(35f, true)
boardId == RayNeoProtocol.BOARD_AIR_4_PRO -> RayNeoSensorProfile(-20f, false)
else -> RayNeoSensorProfile(0f, false)
}
}
}
internal class RayNeoSensorAxes(imuRotationXDeg: Float, private val gt: Boolean) {
private val rotation = Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f),
Math.toRadians((if (gt) 35f else imuRotationXDeg).toDouble()).toFloat())
fun map(v: Vec3): Vec3 = rotation.rotate(if (gt) Vec3(-v.y, v.z, -v.x) else v)
}
internal class RayNeoOrientationFilter(
imuRotationXDeg: Float,
gt: Boolean = false,
private val calibration: RayNeoCalibration? = null,
) {
private val worldUp = Vec3(0f, 1f, 0f)
private val accelGain = 1.5f
private val dpsToRad = (Math.PI.toFloat() / 180f)
private val gravityMps2 = 9.81f
private val stationaryAccelTol = 1.25f
private val stationaryGyroRadPerSec = 0.18f
private val gyroBiasUpdateHz = 0.5f
private val axes = RayNeoSensorAxes(imuRotationXDeg, gt)
private var q = Quaternion.IDENTITY
private var gravityInitialized = false
private var gyroBias = Vec3(0f, 0f, 0f)
private var lastTick100us: Long = -1L
private var lastRealtimeNs: Long = 0L
companion object {
/** Scaffold: right +X, up +Y, back +Z. VITURE: right -Y, up +Z, back -X.
* Conjugate by HeadPose.GLASSES_AXES before entering the unchanged VITURE pose path.
* This is a basis conversion, not a projection or seated-origin adjustment.
*/
fun glassesPose(wxyz: FloatArray): FloatArray =
floatArrayOf(-wxyz[3], -wxyz[1], wxyz[2], wxyz[0])
}
fun update(sample: RayNeoSensorSample): FloatArray {
var accel = Vec3(sample.accelMps2[0], sample.accelMps2[1], sample.accelMps2[2])
var gyro = Vec3(sample.gyroDps[0], sample.gyroDps[1], sample.gyroDps[2])
calibration?.let { gyro -= it.gyroBiasDps }
accel = axes.map(accel)
gyro = axes.map(gyro) * dpsToRad
calibration?.let {
accel = it.level.rotate(accel)
gyro = it.level.rotate(gyro)
}
val nowNs = System.nanoTime()
var dtSec = 0.01f
if (lastTick100us >= 0L && sample.deviceTick100us != lastTick100us) {
dtSec = (((sample.deviceTick100us - lastTick100us) and 0xffffffffL).toFloat() * 1e-4f)
} else if (lastRealtimeNs > 0L && nowNs > lastRealtimeNs) {
dtSec = ((nowNs - lastRealtimeNs).toFloat() * 1e-9f)
}
if (dtSec < 0.001f || dtSec > 0.1f) {
dtSec = 0.01f
}
lastTick100us = sample.deviceTick100us
lastRealtimeNs = nowNs
val accelNorm = accel.length()
val stationary =
kotlin.math.abs(accelNorm - gravityMps2) < stationaryAccelTol &&
gyro.length() < stationaryGyroRadPerSec
if (stationary) {
val alpha = minOf(1f, dtSec * gyroBiasUpdateHz)
gyroBias = gyroBias * (1f - alpha) + gyro * alpha
}
gyro -= gyroBias
if (accelNorm > 1e-3f) {
val measuredUp = accel.normalized()
if (!gravityInitialized) {
// Align gravity before publishing the first pose. Starting at
// identity and settling after recenter can rotate the horizon.
val axis = measuredUp.cross(worldUp)
q = if (measuredUp.y < -0.9999f) {
Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f), Math.PI.toFloat())
} else {
Quaternion(1f + measuredUp.y, axis.x, axis.y, axis.z).normalized()
}
gravityInitialized = true
}
val predictedUp = q.conjugate().rotate(worldUp).normalized()
// q maps body to world: measured x predicted reduces gravity error.
val correction = measuredUp.cross(predictedUp)
gyro += correction * accelGain
}
q = q.integrateBodyRate(gyro, dtSec).normalized()
return floatArrayOf(q.w, q.x, q.y, q.z)
}
}
@@ -0,0 +1,270 @@
// Adapted from informalTechCode/RayNeo-Air-Series-3DoF-Scaffold (Apache-2.0).
// See docs/third-party/rayneo/ for license and protocol attribution.
package com.droiddeck.launcher.glasses
import android.hardware.usb.UsbConstants
import android.hardware.usb.UsbDevice
import android.hardware.usb.UsbEndpoint
import android.hardware.usb.UsbInterface
import java.util.Arrays
private fun Byte.u(): Int = toInt() and 0xFF
data class RayNeoSensorSample(
val accelMps2: FloatArray,
val gyroDps: FloatArray,
val temperatureC: Float,
val magnet: FloatArray,
val proximity: Float,
val light: Float,
val deviceTick100us: Long
)
data class RayNeoDeviceInfo(
val boardId: Int,
val sideBySideEnabled: Boolean
)
sealed interface RayNeoPacket {
data class Sensor(val sample: RayNeoSensorSample) : RayNeoPacket
data class Response(val cmd: Int, val raw: ByteArray) : RayNeoPacket
data object Unknown : RayNeoPacket
}
data class RayNeoEndpointSelection(
val usbInterface: UsbInterface,
val inEndpoint: UsbEndpoint,
val outEndpoint: UsbEndpoint
)
object RayNeoProtocol {
const val VENDOR_ID = 0x1BBB
const val PRODUCT_ID = 0xAF50
// verncat/RayNeo-Air-3S-Pro-OpenVR PR #3 (MIT), tested on GT hardware.
const val GT_VENDOR_ID = 0x3941
private const val GT_CONTROL_INTERFACE = 5
fun isGt(vid: Int, pid: Int) = vid == GT_VENDOR_ID && pid == PRODUCT_ID
const val BOARD_AIR_4_PRO = 0x3A
const val BOARD_GT = 0x40 // Observed on the user's non-Max GT.
const val PKT_MAGIC = 0x99
const val PKT_SENSOR = 0x65
const val PKT_RESPONSE = 0xC8
const val CMD_ACQUIRE_DEVICE_INFO = 0
const val CMD_OPEN_IMU = 1
const val CMD_CLOSE_IMU = 2
const val CMD_SWITCH_TO_3D = 6
const val CMD_SWITCH_TO_2D = 7
private const val RESPONSE_CMD_OFFSET = 8
private const val ACCEL_OFFSET = 4
private const val GYRO_OFFSET = 16
private const val TEMP_OFFSET = 28
private const val MAG_X_OFFSET = 32
private const val TICK_OFFSET = 40
private const val PSENSOR_OFFSET = 44
private const val LSENSOR_OFFSET = 48
private const val MAG_Z_OFFSET = 52
private const val DEVINFO_BOARD_ID_OFFSET = 21
private const val DEVINFO_SIDE_BY_SIDE_OFFSET = 43
fun buildCommandPacket(cmd: Int, arg: Int, packetSize: Int): ByteArray {
val len = maxOf(packetSize, 64)
val pkt = ByteArray(len)
pkt[0] = 0x66
pkt[1] = cmd.toByte()
pkt[2] = arg.toByte()
return pkt
}
fun classifyPacket(raw: ByteArray): RayNeoPacket {
if (raw.size < 4 || raw[0].u() != PKT_MAGIC || raw[2].u() != raw.size) {
return RayNeoPacket.Unknown
}
return when (raw[1].u()) {
PKT_SENSOR -> {
val sample = parseSensorSample(raw)
if (sample != null) RayNeoPacket.Sensor(sample) else RayNeoPacket.Unknown
}
PKT_RESPONSE -> {
if (raw.size <= RESPONSE_CMD_OFFSET) {
RayNeoPacket.Unknown
} else {
RayNeoPacket.Response(raw[RESPONSE_CMD_OFFSET].u(), raw)
}
}
else -> RayNeoPacket.Unknown
}
}
fun parseDeviceInfo(raw: ByteArray): RayNeoDeviceInfo? {
if (raw.size <= DEVINFO_SIDE_BY_SIDE_OFFSET || raw[2].u() != raw.size || raw[0].u() != PKT_MAGIC ||
raw[1].u() != PKT_RESPONSE || raw[RESPONSE_CMD_OFFSET].u() != CMD_ACQUIRE_DEVICE_INFO) {
return null
}
return RayNeoDeviceInfo(
boardId = raw[DEVINFO_BOARD_ID_OFFSET].u(),
sideBySideEnabled = raw[DEVINFO_SIDE_BY_SIDE_OFFSET].u() != 0
)
}
fun selectEndpoints(device: UsbDevice): RayNeoEndpointSelection? {
val gt = isGt(device.vendorId, device.productId)
val gtCandidates = mutableListOf<RayNeoEndpointSelection>()
var fallback: RayNeoEndpointSelection? = null
for (i in 0 until device.interfaceCount) {
val usbInterface = device.getInterface(i)
var inEndpoint: UsbEndpoint? = null
var outEndpoint: UsbEndpoint? = null
for (j in 0 until usbInterface.endpointCount) {
val endpoint = usbInterface.getEndpoint(j)
if (endpoint.type != UsbConstants.USB_ENDPOINT_XFER_INT &&
endpoint.type != UsbConstants.USB_ENDPOINT_XFER_BULK) {
continue
}
// Exclude small button reports from GT's protocol transport.
if (gt && endpoint.maxPacketSize < 64) continue
if (endpoint.direction == UsbConstants.USB_DIR_IN && inEndpoint == null) {
inEndpoint = endpoint
} else if (endpoint.direction == UsbConstants.USB_DIR_OUT && outEndpoint == null) {
outEndpoint = endpoint
}
}
if (inEndpoint != null && outEndpoint != null) {
val selection = RayNeoEndpointSelection(usbInterface, inEndpoint, outEndpoint)
if (gt) {
if (usbInterface.id == GT_CONTROL_INTERFACE) return selection
gtCandidates += selection
continue
}
if (inEndpoint.type == UsbConstants.USB_ENDPOINT_XFER_INT &&
outEndpoint.type == UsbConstants.USB_ENDPOINT_XFER_INT
) {
return selection
}
if (fallback == null) {
fallback = selection
}
}
}
// Interface numbers can change with firmware. Use a unique protocol-sized
// pair when interface 5 is absent, never an arbitrary composite interface.
return if (gt) gtCandidates.singleOrNull() else fallback
}
private fun parseSensorSample(raw: ByteArray): RayNeoSensorSample? {
if (raw.size < MAG_Z_OFFSET + 4 ||
(ACCEL_OFFSET until TEMP_OFFSET step 4).any { !readF32LE(raw, it).isFinite() }) {
return null
}
return RayNeoSensorSample(
accelMps2 = floatArrayOf(
readF32LE(raw, ACCEL_OFFSET + 0),
readF32LE(raw, ACCEL_OFFSET + 4),
readF32LE(raw, ACCEL_OFFSET + 8)
),
gyroDps = floatArrayOf(
readF32LE(raw, GYRO_OFFSET + 0),
readF32LE(raw, GYRO_OFFSET + 4),
readF32LE(raw, GYRO_OFFSET + 8)
),
temperatureC = readF32LE(raw, TEMP_OFFSET),
magnet = floatArrayOf(
readF32LE(raw, MAG_X_OFFSET),
readF32LE(raw, MAG_X_OFFSET + 4),
readF32LE(raw, MAG_Z_OFFSET)
),
proximity = readF32LE(raw, PSENSOR_OFFSET),
light = readF32LE(raw, LSENSOR_OFFSET),
deviceTick100us = readU32LE(raw, TICK_OFFSET).toLong() and 0xffffffffL
)
}
private fun readU32LE(raw: ByteArray, offset: Int): Int {
val b0 = raw[offset].u()
val b1 = raw[offset + 1].u()
val b2 = raw[offset + 2].u()
val b3 = raw[offset + 3].u()
return b0 or (b1 shl 8) or (b2 shl 16) or (b3 shl 24)
}
private fun readF32LE(raw: ByteArray, offset: Int): Float {
return Float.fromBits(readU32LE(raw, offset))
}
}
class RayNeoPacketAssembler {
private val stash = ByteArray(4096)
private var len = 0
fun append(data: ByteArray, count: Int) {
require(count in 0..data.size)
if (count <= 0) {
return
}
if (count >= stash.size || len + count > stash.size) {
len = 0
}
val copyLen = minOf(count, stash.size - len)
System.arraycopy(data, 0, stash, len, copyLen)
len += copyLen
}
fun nextPacket(): ByteArray? {
var offset = 0
while (offset + 4 <= len) {
if (stash[offset].u() != RayNeoProtocol.PKT_MAGIC) {
offset++
continue
}
val packetLen = stash[offset + 2].u()
if (packetLen < 4 || packetLen > 64 ||
stash[offset + 1].u() !in listOf(RayNeoProtocol.PKT_SENSOR, RayNeoProtocol.PKT_RESPONSE)) {
offset++
continue
}
if (offset + packetLen > len) {
break
}
val packet = Arrays.copyOfRange(stash, offset, offset + packetLen)
compact(offset + packetLen)
return packet
}
if (offset > 0) {
compact(offset)
}
return null
}
private fun compact(consumed: Int) {
if (consumed <= 0) {
return
}
if (consumed >= len) {
len = 0
return
}
System.arraycopy(stash, consumed, stash, 0, len - consumed)
len -= consumed
}
}
@@ -0,0 +1,134 @@
package com.droiddeck.launcher.glasses
import android.hardware.usb.UsbConstants
import android.hardware.usb.UsbDevice
import android.hardware.usb.UsbDeviceConnection
import android.hardware.usb.UsbEndpoint
import android.hardware.usb.UsbInterface
import android.hardware.usb.UsbManager
import android.os.SystemClock
import android.util.Log
import java.util.concurrent.atomic.AtomicLong
internal class VitureLink private constructor(
private val connection: UsbDeviceConnection,
private val mcu: UsbInterface,
private val mcuOut: UsbEndpoint,
private val mcuIn: UsbEndpoint?,
private val imu: UsbInterface?,
private val imuIn: UsbEndpoint?,
private val listener: Listener,
) : GlassesLink {
interface Listener {
fun onFrame(frame: VitureProtocol.Frame)
fun onImu(payload: ByteArray, timeNs: Long)
fun onLost()
}
@Volatile private var running = true
private val writeLock = Any()
private var lastSendAt = 0L
private val readers = mutableListOf<Thread>()
private val bad = AtomicLong()
override val hasImu: Boolean get() = imuIn != null
override val badFrames: Long get() = bad.get()
private fun startReaders() {
mcuIn?.let { ep -> readers += reader("viture-mcu", ep) { buf, n -> onPacket(buf, n, false) } }
imuIn?.let { ep ->
readers += reader("viture-imu", ep) { buf, n -> onPacket(buf, n, true) }
}
}
private fun onPacket(buf: ByteArray, size: Int, imuEndpoint: Boolean) {
val frame = VitureProtocol.parse(buf, size) ?: return
if (!frame.crcOk) {
if (bad.incrementAndGet() == 1L) Log.w(TAG, "dropping frames with a bad checksum")
return
}
if (imuEndpoint && frame.magic == VitureProtocol.MAGIC_IMU)
listener.onImu(frame.payload, SystemClock.elapsedRealtimeNanos())
else listener.onFrame(frame)
}
private fun reader(name: String, ep: UsbEndpoint, onPacket: (ByteArray, Int) -> Unit): Thread =
Thread({
val buf = ByteArray(VitureProtocol.PACKET_SIZE)
var failures = 0
while (running) {
val started = SystemClock.uptimeMillis()
val n = connection.bulkTransfer(ep, buf, buf.size, READ_TIMEOUT_MS)
if (!running) break
if (n > 0) {
failures = 0
runCatching { onPacket(buf, n) }.onFailure { Log.w(TAG, "$name packet", it) }
} else if (n < 0 && SystemClock.uptimeMillis() - started < READ_TIMEOUT_MS / 2) {
failures++
SystemClock.sleep(2)
if (failures >= LOST_AFTER) {
Log.w(TAG, "$name: reads keep failing, link lost")
if (running) listener.onLost()
break
}
}
}
}, name).apply { isDaemon = true; priority = Thread.MAX_PRIORITY; start() }
override fun send(msgId: Int, payload: ByteArray): Boolean {
val pkt = VitureProtocol.packet(msgId, payload)
val n = synchronized(writeLock) {
if (!running) return false
val wait = lastSendAt + COMMAND_GAP_MS - SystemClock.uptimeMillis()
if (wait > 0) SystemClock.sleep(wait)
connection.bulkTransfer(mcuOut, pkt, pkt.size, WRITE_TIMEOUT_MS).also { lastSendAt = SystemClock.uptimeMillis() }
}
Log.i(TAG, "send 0x%04X [%s] -> %d".format(msgId, payload.joinToString("") { "%02X".format(it) }, n))
return n == pkt.size
}
override fun close() {
synchronized(writeLock) { running = false }
val self = Thread.currentThread()
readers.forEach { if (it !== self) runCatching { it.join(READ_TIMEOUT_MS * 4L) } }
runCatching { connection.releaseInterface(mcu) }
imu?.let { runCatching { connection.releaseInterface(it) } }
runCatching { connection.close() }
}
companion object {
private const val TAG = "VitureLink"
private const val READ_TIMEOUT_MS = 100
private const val WRITE_TIMEOUT_MS = 250
private const val LOST_AFTER = 50
const val COMMAND_GAP_MS = 60L
fun open(usb: UsbManager, device: UsbDevice, listener: Listener): VitureLink? {
val hid = (0 until device.interfaceCount).map(device::getInterface)
.filter { it.interfaceClass == UsbConstants.USB_CLASS_HID }
.sortedBy { it.id }
for (intf in hid) Log.i(TAG, "iface ${intf.id} eps " + (0 until intf.endpointCount).joinToString {
val ep = intf.getEndpoint(it)
"0x%02X".format(ep.address)
})
val mcu = hid.lastOrNull { it.outEndpoint() != null } ?: return null
val imu = hid.firstOrNull { it.id != mcu.id && it.inEndpoint() != null }
val conn = usb.openDevice(device) ?: return null
if (!conn.claimInterface(mcu, true)) {
conn.close()
return null
}
val imuClaimed = imu != null && conn.claimInterface(imu, true)
val link = VitureLink(conn, mcu, mcu.outEndpoint()!!, mcu.inEndpoint(),
imu.takeIf { imuClaimed }, imu.takeIf { imuClaimed }?.inEndpoint(), listener)
link.startReaders()
return link
}
private fun UsbInterface.outEndpoint(): UsbEndpoint? = (0 until endpointCount).map(::getEndpoint)
.firstOrNull { it.direction == UsbConstants.USB_DIR_OUT && it.type == UsbConstants.USB_ENDPOINT_XFER_INT }
private fun UsbInterface.inEndpoint(): UsbEndpoint? = (0 until endpointCount).map(::getEndpoint)
.firstOrNull { it.direction == UsbConstants.USB_DIR_IN && it.type == UsbConstants.USB_ENDPOINT_XFER_INT }
}
}
@@ -0,0 +1,186 @@
package com.droiddeck.launcher.glasses
import kotlin.math.cos
import kotlin.math.sin
import kotlin.math.sqrt
object VitureProtocol {
const val VENDOR_ID = 0x35CA
const val PACKET_SIZE = 64
private const val HEADER = 0x12
const val MAGIC_COMMAND = 0xFE
const val MAGIC_REPORT = 0xFD
const val MAGIC_IMU = 0xFC
const val MSG_GET_BRIGHTNESS = 0x0005
const val MSG_BRIGHTNESS = 0x0006
const val MSG_GET_DISPLAY = 0x0007
const val MSG_DISPLAY = 0x0008
const val MSG_FILM = 0x000E
const val MSG_IMU = 0x0015
const val MSG_GET_IMU = 0x0017
const val MSG_IMU_RATE = 0x0018
const val MSG_GET_VOLUME = 0x0032
const val MSG_VOLUME = 0x0033
const val EVENT_BRIGHTNESS = 0x0301
const val EVENT_DISPLAY = 0x0302
const val EVENT_VOLUME = 0x0304
const val EVENT_DISPLAY_STEP = 0x030D
const val IMU_RATE_240 = 3
enum class Family { ONE, PRO, LUMA, ULTRA, BEAST, PRO2 }
class Model(override val name: String, val family: Family) : GlassesModel {
override val controllable: Boolean get() = true
val needsSdk: Boolean get() = family == Family.ULTRA || family == Family.BEAST || family == Family.PRO2
val tall: Boolean get() = family == Family.LUMA || family == Family.ULTRA || family == Family.BEAST
override val sbs90: Boolean get() = family == Family.ULTRA
override val brightnessMax: Int get() = 8
override val volumeMax: Int get() = if (family == Family.BEAST) 15 else 8
override val hasSunblock: Boolean get() = family != Family.PRO2
override val diagonalFovDeg: Float get() = when (family) {
Family.ONE -> 43f
Family.PRO -> 46f
Family.PRO2, Family.LUMA, Family.ULTRA -> 50f
Family.BEAST -> 58f
}
override fun mode(hz: Int, sbs: Boolean): Int = when {
tall && sbs -> if (hz >= 90 && sbs90) 0x45 else 0x42
tall -> when { hz >= 120 -> 0x44; hz >= 90 -> 0x43; else -> 0x41 }
sbs -> if (hz >= 90 && sbs90) 0x35 else 0x32
else -> when { hz >= 120 -> 0x34; hz >= 90 -> 0x33; else -> 0x31 }
}
override fun modeForStep(step: Int): Int? = if (tall) null else when (step) {
1 -> 0x31
2 -> 0x33
3 -> 0x34
4 -> 0x32
else -> null
}
}
/** SDK validates the PID before returning this market name. Never guess a new PID. */
fun sdkModel(marketName: String): Model? {
val family = when (marketName) {
"One", "Lite" -> Family.ONE
"Pro" -> Family.PRO
"Pro 2" -> Family.PRO2
"Luma", "Luma Pro", "Luma Cyber" -> Family.LUMA
"Luma Ultra" -> Family.ULTRA
"Beast" -> Family.BEAST
else -> return null
}
return Model("VITURE $marketName", family)
}
fun model(productId: Int, productName: String?): Model? {
val family = when (productId) {
0x1011, 0x1013, 0x1015, 0x1017, 0x101b -> Family.ONE
0x1019, 0x101d -> Family.PRO
0x1121, 0x1131, 0x1141, 0x1151 -> Family.LUMA
0x1101, 0x1104 -> Family.ULTRA
0x1201, 0x1211 -> Family.BEAST
else -> return null
}
val fallback = when (productId) {
0x1015, 0x101b -> "VITURE One Lite"
0x1011, 0x1013, 0x1017 -> "VITURE One"
0x1019 -> "VITURE Pro"
0x101d -> "VITURE Pro XR"
0x1131 -> "VITURE Luma"
0x1121, 0x1141, 0x1151 -> "VITURE Luma Pro"
0x1101, 0x1104 -> "VITURE Luma Ultra"
else -> "VITURE Beast"
}
val name = productName?.trim()?.removeSuffix(" Glasses")?.takeIf { it.startsWith("VITURE", ignoreCase = true) } ?: fallback
return Model(name, family)
}
private val CRC_TABLE = IntArray(256) { i ->
var c = i shl 8
repeat(8) { c = if (c and 0x8000 != 0) (c shl 1) xor 0x1021 else c shl 1 }
c and 0xFFFF
}
fun crc16(data: ByteArray, offset: Int, length: Int): Int {
var crc = 0
for (i in offset until offset + length) {
crc = (CRC_TABLE[((data[i].toInt() and 0xFF) xor (crc ushr 8)) and 0xFF] xor (crc shl 8)) and 0xFFFF
}
return crc
}
fun packet(msgId: Int, payload: ByteArray = ByteArray(0)): ByteArray {
require(payload.size <= PACKET_SIZE - HEADER)
val buf = ByteArray(PACKET_SIZE)
buf[0] = 0xFF.toByte()
buf[1] = MAGIC_COMMAND.toByte()
val len = payload.size + 0x0C
buf[4] = len.toByte()
buf[5] = (len shr 8).toByte()
buf[14] = msgId.toByte()
buf[15] = (msgId shr 8).toByte()
payload.copyInto(buf, HEADER)
val crc = crc16(buf, 4, len + 2)
buf[2] = crc.toByte()
buf[3] = (crc shr 8).toByte()
return buf
}
class Frame(val magic: Int, val msgId: Int, val payload: ByteArray, val crcOk: Boolean)
fun parse(buf: ByteArray, size: Int): Frame? {
if (size < HEADER || size > buf.size || (buf[0].toInt() and 0xFF) != 0xFF) return null
val magic = buf[1].toInt() and 0xFF
if (magic != MAGIC_COMMAND && magic != MAGIC_REPORT && magic != MAGIC_IMU) return null
val len = u16(buf, 4)
if (len < 0x0C || len + 6 > size) return null
val dataLen = len - 0x0C
val crcOk = crc16(buf, 4, len + 2) == u16(buf, 2)
return Frame(magic, u16(buf, 14), buf.copyOfRange(HEADER, HEADER + dataLen), crcOk)
}
private fun u16(buf: ByteArray, at: Int) = (buf[at].toInt() and 0xFF) or ((buf[at + 1].toInt() and 0xFF) shl 8)
fun beFloat(b: ByteArray, at: Int): Float = java.lang.Float.intBitsToFloat(
((b[at].toInt() and 0xFF) shl 24) or ((b[at + 1].toInt() and 0xFF) shl 16) or
((b[at + 2].toInt() and 0xFF) shl 8) or (b[at + 3].toInt() and 0xFF),
)
fun imuQuaternion(payload: ByteArray): FloatArray? {
if (payload.size >= 36) {
val w = beFloat(payload, 20)
val x = beFloat(payload, 24)
val y = beFloat(payload, 28)
val z = beFloat(payload, 32)
val n = sqrt(w * w + x * x + y * y + z * z)
if (n.isFinite() && n > 0.9f && n < 1.1f) return floatArrayOf(x / n, y / n, z / n, w / n)
}
if (payload.size < 12) return null
val roll = beFloat(payload, 0)
val pitch = beFloat(payload, 4)
val yaw = beFloat(payload, 8)
if (!roll.isFinite() || !pitch.isFinite() || !yaw.isFinite()) return null
return eulerZxy(roll, pitch, yaw)
}
fun eulerZxy(rollDeg: Float, pitchDeg: Float, yawDeg: Float): FloatArray {
val r = Math.toRadians(rollDeg.toDouble()) / 2
val p = Math.toRadians(pitchDeg.toDouble()) / 2
val y = Math.toRadians(yawDeg.toDouble()) / 2
val cx = cos(r); val sx = sin(r)
val cy = cos(p); val sy = sin(p)
val cz = cos(y); val sz = sin(y)
return floatArrayOf(
(sx * cy * cz - cx * sy * sz).toFloat(),
(cx * sy * cz + sx * cy * sz).toFloat(),
(cx * cy * sz + sx * sy * cz).toFloat(),
(cx * cy * cz - sx * sy * sz).toFloat(),
)
}
}
@@ -0,0 +1,52 @@
package com.droiddeck.launcher.glasses
import android.util.Log
import kotlin.math.sqrt
/** JNI adapter to VITURE's Android C SDK. The vendor binary is a local build input. */
internal object VitureSdk {
val available: Boolean by lazy {
runCatching {
System.loadLibrary("droiddeck_viture")
nativeAvailable()
}.onFailure { Log.w("VitureSdk", "VITURE driver unavailable", it) }.getOrDefault(false)
}
fun marketName(pid: Int): String? = if (available) nativeMarketName(pid) else null
external fun nativeAvailable(): Boolean
external fun nativeMarketName(pid: Int): String?
external fun nativeOpen(pid: Int, fd: Int, cacheDirectory: String): Long
external fun nativeType(token: Long): Int
external fun nativeTracking(token: Long, enabled: Boolean): Int
external fun nativeCommand(token: Long, command: Int, value: Int): Int
/** Returns CLOCK_BOOTTIME receipt time, zero for no new pose, or a negative SDK error. */
external fun nativePose(token: Long, wxyz: FloatArray): Long
external fun nativeNextState(token: Long, event: IntArray): Int
external fun nativeClose(token: Long)
const val CARINA = 2
const val GET_DISPLAY = 1
const val SET_DISPLAY = 2
const val GET_BRIGHTNESS = 3
const val SET_BRIGHTNESS = 4
const val GET_VOLUME = 5
const val SET_VOLUME = 6
const val SET_FILM = 7
}
internal object VitureSdkPose {
// Pro 2 hardware captures use OpenGL axes despite the generic Gen2 callback's
// NWU documentation. Keep that model-specific exception out of other Gen2 devices.
fun usesOpenGlAxes(carina: Boolean, family: VitureProtocol.Family?) =
carina || family == VitureProtocol.Family.PRO2
/** SDK quaternions are wxyz; normalize their coordinate basis at the boundary. */
fun toGlasses(wxyz: FloatArray, openGlAxes: Boolean): FloatArray? {
if (wxyz.size != 4 || wxyz.any { !it.isFinite() }) return null
val length = sqrt(wxyz.sumOf { it.toDouble() * it }.toFloat())
if (!length.isFinite() || length < 0.5f || length > 1.5f) return null
val q = if (openGlAxes) floatArrayOf(-wxyz[3], -wxyz[1], wxyz[2], wxyz[0])
else floatArrayOf(wxyz[1], wxyz[2], wxyz[3], wxyz[0])
return q.map { it / length }.toFloatArray()
}
}
@@ -0,0 +1,132 @@
package com.droiddeck.launcher.glasses
import android.hardware.usb.UsbDevice
import android.hardware.usb.UsbDeviceConnection
import android.hardware.usb.UsbManager
import android.os.SystemClock
import android.util.Log
import java.io.File
/** SDK owns interface selection (including Ultra bulk endpoints) and all USB I/O. */
internal class VitureSdkLink private constructor(
private val connection: UsbDeviceConnection,
private val token: Long,
private val listener: Listener,
family: VitureProtocol.Family?,
) : GlassesLink {
interface Listener {
fun onValue(msgId: Int, value: Int)
fun onPose(q: FloatArray, timeNs: Long)
fun onLost()
}
private val lock = Any()
@Volatile private var running = true
@Volatile private var tracking = false
override val ready: Boolean get() = running
override val hasImu = true
private val openGlAxes = VitureSdkPose.usesOpenGlAxes(VitureSdk.nativeType(token) == VitureSdk.CARINA, family)
private val worker = Thread({ poll() }, "viture-sdk-pose").apply { isDaemon = true; start() }
override fun send(msgId: Int, payload: ByteArray): Boolean = synchronized(lock) {
if (!running) return false
val value = payload.firstOrNull()?.toInt()?.and(0xff) ?: 0
when (msgId) {
VitureProtocol.MSG_IMU -> {
val wanted = value != 0
if (wanted == tracking) return true
val rc = VitureSdk.nativeTracking(token, wanted)
if (rc == 0 || !wanted) tracking = wanted
return result("tracking", rc)
}
// Frequency is negotiated against SDK capabilities, including Pro 2's pose limit.
VitureProtocol.MSG_IMU_RATE -> return tracking
VitureProtocol.MSG_GET_IMU -> return true
}
val operation = when (msgId) {
VitureProtocol.MSG_GET_DISPLAY -> VitureSdk.GET_DISPLAY
VitureProtocol.MSG_DISPLAY -> VitureSdk.SET_DISPLAY
VitureProtocol.MSG_GET_BRIGHTNESS -> VitureSdk.GET_BRIGHTNESS
VitureProtocol.MSG_BRIGHTNESS -> VitureSdk.SET_BRIGHTNESS
VitureProtocol.MSG_GET_VOLUME -> VitureSdk.GET_VOLUME
VitureProtocol.MSG_VOLUME -> VitureSdk.SET_VOLUME
VitureProtocol.MSG_FILM -> VitureSdk.SET_FILM
else -> return false
}
val rc = VitureSdk.nativeCommand(token, operation, value)
if (rc < 0) return result("command $operation", rc)
if (operation == VitureSdk.GET_DISPLAY || operation == VitureSdk.GET_BRIGHTNESS || operation == VitureSdk.GET_VOLUME)
listener.onValue(msgId, rc)
else if (operation == VitureSdk.SET_DISPLAY) {
val actual = VitureSdk.nativeCommand(token, VitureSdk.GET_DISPLAY, 0)
if (actual >= 0) listener.onValue(VitureProtocol.MSG_GET_DISPLAY, actual)
}
true
}
private fun result(operation: String, rc: Int): Boolean {
if (rc < 0) Log.w("VitureSdkLink", "$operation failed: $rc")
return rc == 0
}
private fun poll() {
val pose = FloatArray(4)
val event = IntArray(2)
var failures = 0
while (running) {
val timestamp = synchronized(lock) {
if (running) repeat(6) {
if (VitureSdk.nativeNextState(token, event) == 1) when (event[0]) {
0 -> listener.onValue(VitureProtocol.MSG_GET_BRIGHTNESS, event[1])
1 -> listener.onValue(VitureProtocol.MSG_GET_VOLUME, event[1])
// A device event can use a different display-mode namespace.
2 -> send(VitureProtocol.MSG_GET_DISPLAY)
3 -> listener.onValue(VitureProtocol.MSG_FILM, event[1])
}
}
if (!running || !tracking) 0L else VitureSdk.nativePose(token, pose)
}
if (timestamp > 0) {
failures = 0
VitureSdkPose.toGlasses(pose, openGlAxes)?.let { listener.onPose(it, timestamp) }
} else if (timestamp == -2L || timestamp == -3L) {
if (++failures >= 10) {
Log.w("VitureSdkLink", "closing after $failures pose USB errors; last SDK result: $timestamp")
listener.onLost()
break
}
}
SystemClock.sleep(if (tracking) 4 else 25)
}
}
override fun close() {
synchronized(lock) {
if (!running) return
running = false
VitureSdk.nativeClose(token)
// Keep Java's descriptor alive until the SDK and its worker threads have stopped.
connection.close()
}
if (Thread.currentThread() !== worker) worker.join(1000)
}
companion object {
fun open(usb: UsbManager, device: UsbDevice, cacheDirectory: File, listener: Listener,
family: VitureProtocol.Family? = null): VitureSdkLink? {
if (!VitureSdk.available) return null
val conn = usb.openDevice(device) ?: return null
val token = runCatching {
cacheDirectory.mkdirs()
VitureSdk.nativeOpen(device.productId, conn.fileDescriptor, cacheDirectory.absolutePath)
}.onFailure { Log.w("VitureSdkLink", "SDK initialization failed", it) }.getOrDefault(0)
if (token <= 0) { conn.close(); return null }
return runCatching { VitureSdkLink(conn, token, listener, family) }.getOrElse {
VitureSdk.nativeClose(token)
conn.close()
Log.w("VitureSdkLink", "SDK worker could not start", it)
null
}
}
}
}
@@ -0,0 +1,88 @@
package com.droiddeck.launcher.glasses
import android.content.Context
import android.net.ConnectivityManager
import android.net.LinkProperties
import android.net.NetworkCapabilities
import java.io.Closeable
import java.io.IOException
import java.io.InputStream
import java.io.OutputStream
import java.net.InetAddress
import java.net.InetSocketAddress
import java.net.Socket
internal interface XrealChannels : Closeable {
val controlInput: InputStream
val controlOutput: OutputStream
val reports: InputStream
}
internal interface XrealConnector : Closeable {
fun connect(): XrealChannels
}
internal class XrealNetworkMissing : IOException("XREAL USB Ethernet network is not available")
/** Bind individual sockets, never the process: Steam must retain its Internet network. */
internal class AndroidXrealConnector(context: Context) : XrealConnector {
private val manager = context.getSystemService(ConnectivityManager::class.java)
private val address = InetAddress.getByName(XrealProtocol.HOST)
private val lock = Any()
private val sockets = mutableSetOf<Socket>()
private var closed = false
companion object {
internal fun hasDirectRoute(properties: LinkProperties, address: InetAddress): Boolean =
properties.routes.any { !it.hasGateway() && it.destination.prefixLength > 0 && it.matches(address) }
}
override fun connect(): XrealChannels {
val network = manager.allNetworks.firstOrNull { network ->
manager.getNetworkCapabilities(network)?.let {
it.hasTransport(NetworkCapabilities.TRANSPORT_ETHERNET) &&
!it.hasTransport(NetworkCapabilities.TRANSPORT_VPN)
} == true && manager.getLinkProperties(network)?.let { hasDirectRoute(it, address) } == true
} ?: throw XrealNetworkMissing()
fun open(port: Int): Socket {
val socket = Socket()
synchronized(lock) {
if (closed) { socket.close(); throw IOException("XREAL connection closed") }
sockets.add(socket)
}
try {
network.bindSocket(socket)
socket.soTimeout = 2000
socket.tcpNoDelay = true
socket.connect(InetSocketAddress(address, port), 1500)
return socket
} catch (e: Exception) {
release(socket)
throw e
}
}
val control = open(XrealProtocol.CONTROL_PORT)
val reportSocket = try { open(XrealProtocol.REPORT_PORT) } catch (e: Exception) {
release(control)
throw e
}
return object : XrealChannels {
override val controlInput = control.getInputStream()
override val controlOutput = control.getOutputStream()
override val reports = reportSocket.getInputStream()
override fun close() { release(control); release(reportSocket) }
}
}
private fun release(socket: Socket) = synchronized(lock) {
runCatching { socket.close() }
sockets.remove(socket)
Unit
}
override fun close() = synchronized(lock) {
closed = true
sockets.forEach { runCatching { it.close() } }
sockets.clear()
}
}
@@ -0,0 +1,201 @@
package com.droiddeck.launcher.glasses
import android.content.Context
import android.util.Log
import java.io.IOException
import java.util.concurrent.ArrayBlockingQueue
import java.util.concurrent.TimeUnit
import java.util.concurrent.atomic.AtomicReference
/** One connection owns two TCP channels. All control requests are serialized off the UI thread. */
internal class XrealLink internal constructor(
private val connector: XrealConnector,
private val listener: Listener,
) : GlassesLink {
interface Listener {
fun onReady()
fun onValue(message: Int, value: Int)
fun onPose(raw: ByteArray, pose: FloatArray, timeNs: Long)
fun onUnavailable(networkMissing: Boolean)
}
private data class Command(val message: Int, val value: Int)
private val commands = ArrayBlockingQueue<Command>(16)
private val connection = AtomicReference<XrealChannels?>()
@Volatile private var running = true
@Volatile private var tracking = false
@Volatile override var ready = false
private set
override val hasImu = true
private val worker = Thread(::run, "xreal-control").apply { isDaemon = true; start() }
companion object {
fun open(context: Context, listener: Listener) = XrealLink(AndroidXrealConnector(context), listener)
}
override fun send(msgId: Int, payload: ByteArray): Boolean {
if (!running || !ready) return false
val value = payload.firstOrNull()?.toInt()?.and(255) ?: -1
if (msgId == VitureProtocol.MSG_IMU) {
if (value !in 0..1) return false
tracking = value == 1
return true
}
if (msgId == VitureProtocol.MSG_GET_IMU) return true
val valid = when (msgId) {
VitureProtocol.MSG_GET_DISPLAY -> true
VitureProtocol.MSG_DISPLAY -> value in 2..5 || value == XrealModel.NATIVE_60HZ
VitureProtocol.MSG_BRIGHTNESS -> value in 0..9
VitureProtocol.MSG_FILM -> value in 0..1
else -> false // No guessed volume, sensor-rate, or VITURE wire commands.
}
return valid && commands.offer(Command(msgId, value))
}
private fun run() {
var lastError: String? = null
while (running) {
var channels: XrealChannels? = null
var reports: Thread? = null
val failure = AtomicReference<Exception?>()
try {
channels = connector.connect()
connection.set(channels)
if (!running) break
val control = XrealControl(channels)
listener.onValue(VitureProtocol.MSG_GET_DISPLAY, control.display())
val current = channels
reports = Thread({
try {
var filter = XrealOrientationFilter()
var wasTracking = false
var lastSensor = System.nanoTime()
while (running && connection.get() === current) {
val frame = XrealProtocol.read(current.reports)
val sample = XrealProtocol.sample(frame)
if (sample != null) {
lastSensor = System.nanoTime()
val enabled = tracking
if (enabled && !wasTracking) filter = XrealOrientationFilter()
wasTracking = enabled
if (enabled) filter.update(sample)?.let {
if (running && ready && tracking && connection.get() === current)
listener.onPose(frame.body, it, System.nanoTime())
}
} else if (System.nanoTime() - lastSensor > 3_000_000_000L) {
throw IOException("No valid XREAL IMU reports")
}
}
} catch (e: Exception) {
failure.set(e)
current.close() // Wake a control read immediately.
}
}, "xreal-reports").apply { isDaemon = true; start() }
ready = true
lastError = null
listener.onReady()
var heartbeat = System.nanoTime()
while (running && failure.get() == null) {
val command = commands.poll(250, TimeUnit.MILLISECONDS)
if (command != null) {
when (command.message) {
VitureProtocol.MSG_GET_DISPLAY ->
listener.onValue(command.message, control.display())
VitureProtocol.MSG_DISPLAY -> {
listener.onValue(VitureProtocol.MSG_GET_DISPLAY, control.changeDisplay(command.value))
}
VitureProtocol.MSG_BRIGHTNESS -> {
control.set(XrealProtocol.SET_BRIGHTNESS, command.value)
listener.onValue(command.message, command.value)
}
VitureProtocol.MSG_FILM -> {
control.set(XrealProtocol.SET_DIMMER, if (command.value == 1) 2 else 0)
listener.onValue(command.message, command.value)
}
}
}
if (System.nanoTime() - heartbeat > 5_000_000_000L) {
listener.onValue(VitureProtocol.MSG_GET_DISPLAY, control.display())
heartbeat = System.nanoTime()
}
}
failure.get()?.let { throw it }
} catch (e: Exception) {
ready = false
if (running && lastError != e.message) {
lastError = e.message
Log.w("XrealLink", "Connection unavailable: ${e.message}")
listener.onUnavailable(e is XrealNetworkMissing)
}
} finally {
ready = false
tracking = false
commands.clear()
connection.compareAndSet(channels, null)
channels?.close()
try { reports?.join(2500) } catch (_: InterruptedException) { }
}
if (running) try { Thread.sleep(2000) } catch (_: InterruptedException) { }
}
}
override fun close() {
running = false
ready = false
tracking = false
commands.clear()
connection.getAndSet(null)?.close()
connector.close() // Includes sockets still inside connect().
worker.interrupt()
}
}
/** Kept independent of Android so the real framing/acknowledgement path is testable. */
internal class XrealControl(private val channels: XrealChannels, private val settleDelayMs: Long = 150) {
private var transaction = 0
private fun exchange(command: Int, value: Int? = null): ByteArray {
transaction = if (transaction == Int.MAX_VALUE) 1 else transaction + 1
channels.controlOutput.write(XrealProtocol.request(command, transaction, value))
channels.controlOutput.flush()
val deadline = System.nanoTime() + 3_000_000_000L
repeat(32) { // Bound unsolicited button/status packets and stale responses.
if (System.nanoTime() > deadline) throw IOException("XREAL control timed out")
XrealProtocol.response(XrealProtocol.read(channels.controlInput), command, transaction)?.let { return it }
}
throw IOException("XREAL control response was not received")
}
fun display(): Int {
val reply = exchange(XrealProtocol.GET_DISPLAY)
return XrealProtocol.numeric(reply)?.takeIf { it in 2..5 || it == XrealModel.NATIVE_60HZ }
?: throw IOException("Unsupported XREAL display response: ${reply.take(32).joinToString("") { "%02x".format(it) }}")
}
fun changeDisplay(mode: Int): Int {
val stereo = mode == 5
set(XrealProtocol.SET_INPUT, if (stereo) 1 else 0)
// On 1S, SET_INPUT itself changes EDID. Sending SET_DISPLAY immediately
// receives error 5001 while that change is pending. GET_DISPLAY returns
// an empty property during the transition; wait for a real mode report.
val current = awaitDisplay { (it == 5) == stereo }
if (current == mode) return current
set(XrealProtocol.SET_DISPLAY, mode)
return awaitDisplay { it == mode }
}
private fun awaitDisplay(matches: (Int) -> Boolean): Int {
val deadline = System.nanoTime() + 8_000_000_000L
repeat(40) {
val reply = exchange(XrealProtocol.GET_DISPLAY)
val value = XrealProtocol.numeric(reply)
if (value != null && matches(value)) return value
if (value == null && !XrealProtocol.acknowledged(reply))
throw IOException("Invalid XREAL display transition response")
if (System.nanoTime() > deadline) throw IOException("XREAL display transition timed out")
Thread.sleep(settleDelayMs)
}
throw IOException("XREAL display transition did not complete")
}
fun set(command: Int, value: Int) {
val reply = exchange(command, value)
if (!XrealProtocol.acknowledged(reply)) throw IOException("XREAL command %04x was not acknowledged: %s".format(
command, reply.take(32).joinToString("") { "%02x".format(it) }))
}
}
@@ -0,0 +1,27 @@
package com.droiddeck.launcher.glasses
/** First One-family target. Recovery and Air USB products must not use this transport. */
object XrealModel : GlassesModel {
const val VENDOR_ID = 0x3318
const val PRODUCT_1S = 0x043e
const val NATIVE_60HZ = 29 // Captured on 1S: 1920 x 1200 at 60 Hz.
override val name = "XREAL 1S"
override val controllable = true
override val brightnessMax = 9
override val volumeMax = 0
override val hasSunblock = true
override val diagonalFovDeg = 52f
override val sbs90 = false
fun matches(vid: Int, pid: Int) = vid == VENDOR_ID && pid == PRODUCT_1S
fun isDisplay(name: String) = name.contains("XREAL", ignoreCase = true)
// One-family EDID configuration values, not VITURE's display mode IDs.
override fun mode(hz: Int, sbs: Boolean) = when {
sbs -> 5
hz >= 120 -> 4
hz >= 90 -> 3
else -> NATIVE_60HZ
}
override fun modeForStep(step: Int): Int? = null
}
@@ -0,0 +1,46 @@
package com.droiddeck.launcher.glasses
import kotlin.math.abs
/** 1S radians/sec and device nanoseconds, mapped using directed hardware captures. */
internal class XrealOrientationFilter {
private var previous = 0L
private var rotation = Quaternion.IDENTITY
private var initialized = false
private var bias = Vec3(0f, 0f, 0f)
fun update(sample: XrealProtocol.Sample): FloatArray? {
if (sample.timeNs <= previous) return null
val dt = if (previous == 0L) 0f else (sample.timeNs - previous) * 1e-9f
previous = sample.timeNs
// 1S reports right/down/forward. Rotate 180 degrees about X into the
// shared right/up/back basis before either gravity alignment or fusion.
// Captures: level gravity -Y; looking left -gyroY; up +gyroX; right tilt +gyroZ.
val accel = Vec3(sample.accel.x, -sample.accel.y, -sample.accel.z)
val gyro = Vec3(sample.gyro.x, -sample.gyro.y, -sample.gyro.z)
val norm = accel.length()
val gravityValid = abs(norm - 9.81f) < 1.5f
if (dt > 0.25f) initialized = false // Never integrate a reconnect/stall as motion.
if (!initialized) {
if (!gravityValid) return null
val up = accel.normalized()
val axis = up.cross(Vec3(0f, 1f, 0f))
rotation = if (up.y < -0.9999f) Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f), Math.PI.toFloat())
else Quaternion(1f + up.y, axis.x, axis.y, axis.z).normalized()
initialized = true
} else if (dt in 0.0001f..0.05f) {
if (gravityValid && gyro.length() < 0.04f) {
val alpha = (dt * 0.5f).coerceAtMost(1f)
bias = bias * (1f - alpha) + gyro * alpha
}
var rate = gyro - bias
if (gravityValid) {
val predictedUp = rotation.conjugate().rotate(Vec3(0f, 1f, 0f))
rate += accel.normalized().cross(predictedUp) * 1.5f
}
rotation = rotation.integrateBodyRate(rate, dt).normalized()
}
// Same right/up/back -> shared glasses basis used by the RayNeo adapter.
return RayNeoOrientationFilter.glassesPose(floatArrayOf(rotation.w, rotation.x, rotation.y, rotation.z))
}
}
@@ -0,0 +1,78 @@
package com.droiddeck.launcher.glasses
import java.io.EOFException
import java.io.IOException
import java.io.InputStream
import java.nio.ByteBuffer
import java.nio.ByteOrder
/** Independent implementation of the One-family wire format; see xreal-investigation.md. */
internal object XrealProtocol {
const val HOST = "169.254.2.1"
const val REPORT_PORT = 52998
const val CONTROL_PORT = 52999
const val REPORT = 0x2836
const val GET_DISPLAY = 0x275e
const val SET_DISPLAY = 0x275f
const val SET_INPUT = 0x2822
const val SET_BRIGHTNESS = 0x271c
const val SET_DIMMER = 0x2727
const val MAX_BODY = 64 * 1024
data class Frame(val command: Int, val body: ByteArray)
data class Sample(val timeNs: Long, val gyro: Vec3, val accel: Vec3)
fun read(input: InputStream): Frame {
val deadline = System.nanoTime() + 3_000_000_000L
val header = ByteArray(6)
input.readFully(header, deadline)
val size = ByteBuffer.wrap(header, 2, 4).int
if (size !in 0..MAX_BODY) throw IOException("Invalid XREAL frame length: $size")
return Frame(((header[0].toInt() and 255) shl 8) or (header[1].toInt() and 255),
ByteArray(size).also { input.readFully(it, deadline) })
}
private fun InputStream.readFully(bytes: ByteArray, deadline: Long) {
var at = 0
while (at < bytes.size) {
if (System.nanoTime() > deadline) throw IOException("XREAL frame timed out")
val count = read(bytes, at, bytes.size - at)
if (count < 0) throw EOFException("XREAL stream closed mid-frame")
if (count == 0) throw IOException("XREAL stream made no progress")
at += count
}
}
fun sample(frame: Frame): Sample? {
if (frame.command != REPORT || frame.body.size != 128) return null
val b = ByteBuffer.wrap(frame.body).order(ByteOrder.LITTLE_ENDIAN)
if (b.getInt(24) != 11) return null // Magnetometer reports are a different type.
val time = b.getLong(8)
val gyro = Vec3(b.getFloat(28), b.getFloat(32), b.getFloat(36))
val accel = Vec3(b.getFloat(40), b.getFloat(44), b.getFloat(48))
if (time <= 0 || !gyro.valid(100f) || !accel.valid(200f)) return null
return Sample(time, gyro, accel)
}
private fun Vec3.valid(limit: Float) = listOf(x, y, z).all { it.isFinite() && kotlin.math.abs(it) <= limit }
fun request(command: Int, transaction: Int, value: Int? = null): ByteArray {
require(transaction > 0)
require(value == null || value in 0..127)
val payload = if (value == null) byteArrayOf(0x18, 0) else byteArrayOf(0x1a, 2, 8, value.toByte())
return ByteBuffer.allocate(10 + payload.size).putShort(command.toShort())
.putInt(4 + payload.size).putInt(transaction or Int.MIN_VALUE).put(payload).array()
}
/** No acknowledgement is inferred from a write; match both command and transaction. */
fun response(frame: Frame, command: Int, transaction: Int): ByteArray? {
if (frame.command != command || frame.body.size < 4 ||
ByteBuffer.wrap(frame.body).int != transaction) return null
return frame.body.copyOfRange(4, frame.body.size)
}
fun acknowledged(payload: ByteArray) = payload.contentEquals(byteArrayOf(0x22, 0))
fun numeric(payload: ByteArray): Int? = if (payload.size == 4 &&
payload[0] == 0x22.toByte() && payload[1] == 2.toByte() && payload[2] == 0x10.toByte() &&
payload[3].toInt() in 0..127) payload[3].toInt() else null
}
@@ -12,13 +12,15 @@ data class Cardboard(
val screenWidthDm: Int = 40,
val screenDistanceDm: Int = 30,
val headTracking: Boolean = true,
val mono: Boolean = false,
) {
val lens: Viewer get() = Viewer.byId(viewer)
fun push(panelWidthMm: Float, panelHeightMm: Float) {
fun push(panelWidthMm: Float, panelHeightMm: Float, fovOverrideDeg: Float = 0f, displayHz: Float = 0f) {
val l = lens
WaylandCompositor.nativeSetCardboard(enabled, source, ipdMm / 1000f, screenDistanceDm / 10f,
screenWidthDm / 10f, l.k1, l.k2, fovDeg(l, panelHeightMm), lensInset(panelWidthMm, l.lensSeparationMm), l.chroma)
screenWidthDm / 10f, l.k1, l.k2, if (fovOverrideDeg > 0f) fovOverrideDeg else fovDeg(l, panelHeightMm),
if (mono) 0f else lensInset(panelWidthMm, l.lensSeparationMm), l.chroma, mono, displayHz)
}
fun encode(): JSONObject = JSONObject()
@@ -29,7 +31,8 @@ data class Cardboard(
val lensSeparationMm: Float, val chroma: Float, val fallbackFovDeg: Float) {
CARDBOARD_V2("cardboard2", 0.34f, 0.55f, 39.3f, 63.9f, 0.006f, 90f),
CARDBOARD_V1("cardboard1", 0.441f, 0.156f, 42f, 60f, 0.004f, 80f),
NONE("none", 0f, 0f, 0f, 0f, 0f, 60f);
NONE("none", 0f, 0f, 0f, 0f, 0f, 60f),
VITURE("viture", 0f, 0f, 0f, 0f, 0f, 23.3f);
companion object {
fun byId(id: String): Viewer = entries.firstOrNull { it.id == id } ?: CARDBOARD_V2
@@ -123,6 +126,10 @@ object HeadPose {
a[r * 3] * b[c] + a[r * 3 + 1] * b[3 + c] + a[r * 3 + 2] * b[6 + c]
}
private val GLASSES_AXES = floatArrayOf(0f, 0f, -1f, -1f, 0f, 0f, 0f, 1f, 0f)
fun fromGlasses(x: Float, y: Float, z: Float, w: Float): FloatArray = multiply(fromQuaternion(x, y, z, w), GLASSES_AXES)
fun transpose(m: FloatArray): FloatArray = floatArrayOf(m[0], m[3], m[6], m[1], m[4], m[7], m[2], m[5], m[8])
fun apply(m: FloatArray, v: FloatArray): FloatArray = floatArrayOf(
@@ -10,19 +10,42 @@ import android.os.HandlerThread
import android.util.Log
import com.droiddeck.launcher.wayland.WaylandCompositor
class HeadTracker(context: Context, private val rotation: () -> Int) : SensorEventListener {
class HeadTracker(context: Context, private val glasses: Boolean = false, private val rotation: () -> Int) : SensorEventListener {
private val sensors = context.getSystemService(Context.SENSOR_SERVICE) as SensorManager
private val sensor: Sensor? = sensors.getDefaultSensor(Sensor.TYPE_GAME_ROTATION_VECTOR)
?: sensors.getDefaultSensor(Sensor.TYPE_ROTATION_VECTOR)
private var thread: HandlerThread? = null
private var started = false
private val glassesLock = Any()
@Volatile private var reference: HeadPose.Reference? = null
@Volatile private var recenterPending = true
@Volatile private var latest: FloatArray? = null
private val glassesSink = com.droiddeck.launcher.glasses.Glasses.PoseSink { x, y, z, w, _ ->
val m = HeadPose.fromGlasses(x, y, z, w)
synchronized(glassesLock) {
if (!started) return@PoseSink
latest = m
apply(m)
}
}
val usesGlasses: Boolean get() = glasses
val available: Boolean get() = sensor != null
val sensorName: String get() = sensor?.let { "${it.name} (${if (it.type == Sensor.TYPE_GAME_ROTATION_VECTOR) "game rotation vector" else "rotation vector"})" } ?: "none"
fun start() {
if (glasses) {
synchronized(glassesLock) {
if (started) return
started = true
recenterPending = true
}
com.droiddeck.launcher.glasses.Glasses.setPoseSink(glassesSink)
WaylandCompositor.nativeLog("cardboard", "head tracking on: VITURE glasses IMU")
return
}
val s = sensor ?: run {
WaylandCompositor.nativeLog("cardboard", "no rotation sensor: the virtual screen stays head-locked")
WaylandCompositor.nativeSetCardboardPose(HeadPose.IDENTITY, false)
@@ -37,6 +60,15 @@ class HeadTracker(context: Context, private val rotation: () -> Int) : SensorEve
}
fun stop() {
if (glasses) {
synchronized(glassesLock) {
if (!started) return
started = false
WaylandCompositor.nativeSetCardboardPose(HeadPose.IDENTITY, false)
}
com.droiddeck.launcher.glasses.Glasses.clearPoseSink(glassesSink)
return
}
val t = thread ?: return
sensors.unregisterListener(this)
t.quitSafely()
@@ -46,7 +78,8 @@ class HeadTracker(context: Context, private val rotation: () -> Int) : SensorEve
fun recenter() {
recenterPending = true
latest?.let { apply(it) }
if (glasses) synchronized(glassesLock) { if (started) latest?.let { apply(it) } }
else latest?.let { apply(it) }
}
override fun onSensorChanged(event: SensorEvent) {
@@ -154,7 +154,9 @@ object Lossless {
private fun steamDll(context: Context): File? {
val libraries = listOfNotNull(
File(LinuxRuntime.rootDir(context), "root/.local/share/Steam"),
File(LinuxRuntime.rootDir(context), "root/.droiddeck-x64/.local/share/Steam"),
GameStorage.effective(context)?.path?.let(::File),
GameStorage.x64CardLibrary(context),
) + com.droiddeck.launcher.frontend.AddedGames.roots(context).map { it.host }
return libraries.map { File(it, STEAM_DLL) }.filter { it.isFile }.maxByOrNull { it.lastModified() }
}
@@ -5,6 +5,8 @@ import com.droiddeck.launcher.agent.AgentGuest
import org.json.JSONObject
object SteamVr {
const val APP_ID = "250820"
enum class Status {
UNAVAILABLE,
NOT_INSTALLED,
@@ -13,6 +15,23 @@ object SteamVr {
RUNNING,
}
internal fun shouldPoll(status: Status?, cardboardEnabled: Boolean, pictureSwitched: Boolean,
glassesControl: Boolean, requested: Boolean): Boolean =
requested || cardboardEnabled || pictureSwitched || glassesControl ||
status == Status.RUNNING || status == Status.INSTALLING
internal fun autoAction(
status: Status, requested: Boolean, loading: Boolean, steamSession: Boolean,
startTried: Boolean, installTried: Boolean,
): String? {
if (!requested || loading || !steamSession || startTried) return null
return when (status) {
Status.READY -> "start"
Status.NOT_INSTALLED -> if (installTried) null else "install"
else -> null
}
}
fun status(context: Context): Status = runCatching {
parse(AgentGuest.call(context, JSONObject().put("kind", "vr").put("action", "status"), STATUS_TIMEOUT_MS))
}.getOrDefault(Status.UNAVAILABLE)
@@ -0,0 +1,86 @@
package com.droiddeck.launcher.gpu
import org.json.JSONObject
import java.io.File
import java.io.RandomAccessFile
import java.nio.file.Files
import java.nio.file.attribute.BasicFileAttributes
/** Only new compositor output is evidence of a fault in the current session. */
internal class SteamVrDeviceLossLog(private val file: File) {
private var identity: Any? = identity()
private var position = file.length()
private var prefix = runCatching {
RandomAccessFile(file, "r").use { input -> ByteArray(minOf(64, input.length()).toInt()).also(input::readFully) }
}.getOrDefault(byteArrayOf())
private var partial = ""
private var handled = false
private fun identity(): Any? = runCatching {
Files.readAttributes(file.toPath(), BasicFileAttributes::class.java).fileKey()
}.getOrNull()
fun poll(): Boolean {
if (handled) return true
if (!file.isFile) return false
return runCatching {
val current = identity()
RandomAccessFile(file, "r").use { input ->
val head = ByteArray(minOf(64, input.length()).toInt()).also(input::readFully)
val replaced = prefix.isNotEmpty() && !head.take(prefix.size).toByteArray().contentEquals(prefix)
if (current != identity || input.length() < position || replaced) {
position = 0
partial = ""
}
identity = current
prefix = head
// Fault loops can produce megabytes per second. Bound work and keep the newest output.
if (input.length() - position > MAX_READ) {
position = input.length() - MAX_READ
partial = ""
}
input.seek(position)
val bytes = ByteArray(minOf(MAX_READ, input.length() - position).toInt())
input.readFully(bytes)
position = input.filePointer
val text = partial + bytes.toString(Charsets.UTF_8)
val end = text.lastIndexOf('\n')
partial = text.substring(end + 1).takeLast(2048)
if (end >= 0 && isDeviceLoss(text.substring(0, end + 1))) handled = true
handled
}
}.getOrDefault(false)
}
companion object {
private const val MAX_READ = 128 * 1024L
internal fun isDeviceLoss(text: String): Boolean =
Regex("\\bVK_ERROR_DEVICE_LOST\\b|\\bvkerror\\s*=\\s*-4\\b", RegexOption.IGNORE_CASE).containsMatchIn(text)
}
}
internal object SteamVrRecoveryPolicy {
const val COOLDOWN_MS = 10 * 60 * 1000L
const val STOP_MS = 15_000L
const val LOGS_MS = 30_000L
private const val POLL_MS = 250L
enum class Shutdown { COMPLETE, LOGS_PENDING, STUCK }
fun canRestartAutomatically(lastAttempt: Long, now: Long): Boolean =
lastAttempt == 0L || (now >= lastAttempt && now - lastAttempt >= COOLDOWN_MS)
fun awaitShutdown(session: () -> JSONObject?, now: () -> Long, sleep: (Long) -> Unit): Shutdown {
fun await(ms: Long, done: (JSONObject) -> Boolean): Boolean {
val deadline = now() + ms
while (true) {
if (session()?.let(done) == true) return true
if (now() >= deadline) return false
sleep(POLL_MS)
}
}
if (!await(STOP_MS) { it.optString("phase") in setOf("IDLE", "FAILED") }) return Shutdown.STUCK
return if (await(LOGS_MS) { !it.optBoolean("artifactsAvailable") || it.optBoolean("artifactsComplete") }) Shutdown.COMPLETE
else Shutdown.LOGS_PENDING
}
}
@@ -0,0 +1,83 @@
package com.droiddeck.launcher.gpu
import android.app.Activity
import android.app.AlertDialog
import android.content.Context
import android.content.Intent
import android.os.Handler
import android.os.Looper
import com.droiddeck.launcher.R
import com.droiddeck.launcher.runtime.LinuxRuntime
import com.droiddeck.launcher.session.SessionEvents
import java.io.File
/** Process-wide cursor survives Activity recreation; a new session ignores older log errors. */
internal object SteamVrHealth {
private var session: String? = null
private var log: SteamVrDeviceLossLog? = null
@Synchronized fun begin(context: Context, sessionId: String?) {
if (sessionId == session && log != null) return
session = sessionId
log = SteamVrDeviceLossLog(File(LinuxRuntime.rootDir(context),
"root/.local/share/Steam/logs/vrcompositor.txt"))
}
@Synchronized fun deviceLost(): Boolean = log?.poll() == true
}
internal class SteamVrRecovery(private val activity: Activity, private val foreground: () -> Boolean) {
private val handler = Handler(Looper.getMainLooper())
private var dialog: AlertDialog? = null
private var handled = false
private var countdown: Runnable? = null
fun detected() {
if (handled || !foreground() || activity.isFinishing || activity.isDestroyed) return
handled = true
SessionEvents.record("steamvr.device_lost")
val prefs = activity.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
val automatic = SteamVrRecoveryPolicy.canRestartAutomatically(prefs.getLong(LAST_ATTEMPT, 0), System.currentTimeMillis())
var seconds = 8
fun message() = if (automatic) activity.getString(R.string.steamvr_recovery_countdown, seconds)
else activity.getString(R.string.steamvr_recovery_repeated)
val prompt = AlertDialog.Builder(activity)
.setTitle(R.string.steamvr_recovery_title)
.setMessage(message())
.setPositiveButton(R.string.steamvr_recovery_restart) { _, _ -> restart() }
.setNegativeButton(R.string.steamvr_recovery_cancel) { _, _ -> }
.create()
prompt.setOnDismissListener { countdown?.let(handler::removeCallbacks); countdown = null; dialog = null }
dialog = prompt
// A touch meant for the game must not silently dismiss the recovery controls.
prompt.setCanceledOnTouchOutside(false)
prompt.show()
if (automatic) {
countdown = object : Runnable {
override fun run() {
if (!foreground() || activity.isFinishing || activity.isDestroyed) { handled = false; prompt.dismiss(); return }
if (--seconds <= 0) { prompt.dismiss(); restart(); return }
prompt.setMessage(message())
handler.postDelayed(this, 1000)
}
}.also { handler.postDelayed(it, 1000) }
}
}
private fun restart() {
if (!foreground()) return
// Commit before crossing processes so a repeated fault cannot create a restart loop.
if (!activity.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit()
.putLong(LAST_ATTEMPT, System.currentTimeMillis()).commit()) return
SessionEvents.record("steamvr.recovery_requested")
activity.startActivity(Intent(activity, SteamVrRestartActivity::class.java)
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))
}
fun close() { countdown?.let(handler::removeCallbacks); countdown = null; dialog?.dismiss() }
companion object {
internal const val PREFS = "steamvr_recovery"
internal const val LAST_ATTEMPT = "lastAttempt"
}
}
@@ -0,0 +1,27 @@
package com.droiddeck.launcher.gpu
import android.util.AtomicFile
import org.json.JSONObject
import java.io.File
internal object SteamVrRecoverySettings {
/** After a graphics crash SteamVR can quarantine our HMD, leaving a server with no VR output. */
fun unblockHeadset(file: File): Boolean {
if (!file.isFile) return false
val settings = JSONObject(file.readText())
val driver = settings.optJSONObject("driver_droiddeck") ?: return false
if (!driver.optBoolean("blocked_by_safe_mode")) return false
// Preserve every other setting, including safe mode and blocks on other drivers.
driver.put("blocked_by_safe_mode", false)
val atomic = AtomicFile(file)
val output = atomic.startWrite()
try {
output.write(settings.toString(3).toByteArray(Charsets.UTF_8))
atomic.finishWrite(output)
} catch (e: Exception) {
atomic.failWrite(output)
throw e
}
return true
}
}
@@ -0,0 +1,72 @@
package com.droiddeck.launcher.gpu
import android.app.Activity
import android.app.ActivityManager
import android.content.Intent
import android.net.Uri
import android.os.Bundle
import android.os.Process
import android.os.SystemClock
import android.view.Gravity
import android.widget.TextView
import com.droiddeck.launcher.R
import com.droiddeck.launcher.SessionActivity
import com.droiddeck.launcher.session.SessionService
import org.json.JSONObject
/** A visible, private process survives destruction of the app's native graphics context. */
class SteamVrRestartActivity : Activity() {
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(TextView(this).apply {
setText(R.string.steamvr_recovery_restarting)
gravity = Gravity.CENTER
textSize = 22f
})
Thread({ recover() }, "steamvr-process-recovery").start()
}
private fun recover() {
try {
val manager = getSystemService(ACTIVITY_SERVICE) as ActivityManager
val main = manager.runningAppProcesses?.firstOrNull {
it.processName == packageName && it.uid == Process.myUid() && it.pid != Process.myPid()
} ?: error("DroidDeck process is unavailable")
val uri = Uri.parse("content://$packageName.agent")
contentResolver.call(uri, "stop", null, null)
val shutdown = SteamVrRecoveryPolicy.awaitShutdown({
runCatching {
contentResolver.call(uri, "state", null, null)?.getString("json")?.let(::JSONObject)?.optJSONObject("session")
}.getOrNull()
}, SystemClock::elapsedRealtime, SystemClock::sleep)
if (shutdown != SteamVrRecoveryPolicy.Shutdown.COMPLETE)
android.util.Log.w("SteamVrRecovery", "Restarting before the session finished stopping: $shutdown")
// Only the verified main process, never a caller-supplied PID or an external app.
Process.killProcess(main.pid)
val exited = SystemClock.elapsedRealtime() + 5000
while (java.io.File("/proc/${main.pid}").exists() && SystemClock.elapsedRealtime() < exited) SystemClock.sleep(50)
check(!java.io.File("/proc/${main.pid}").exists()) { "DroidDeck process did not exit" }
val settings = java.io.File(com.droiddeck.launcher.runtime.LinuxRuntime.rootDir(this),
"root/.local/share/Steam/config/steamvr.vrsettings")
runCatching { SteamVrRecoverySettings.unblockHeadset(settings) }
.onSuccess { if (it) android.util.Log.i("SteamVrRecovery", "Cleared the DroidDeck HMD crash block before restarting SteamVR") }
.onFailure { android.util.Log.w("SteamVrRecovery", "Could not read the SteamVR settings; restarting with them unchanged", it) }
runOnUiThread {
startActivity(Intent(this, SessionActivity::class.java)
.putExtra(SessionService.EXTRA_MODE, SessionService.MODE_STEAM)
.putExtra(SessionActivity.EXTRA_STEAMVR, true)
.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_CLEAR_TASK))
finishAndRemoveTask()
}
} catch (e: Exception) {
android.util.Log.e("SteamVrRecovery", "Could not restart DroidDeck", e)
runOnUiThread {
setContentView(TextView(this).apply {
setText(R.string.steamvr_recovery_failed)
gravity = Gravity.CENTER
textSize = 22f
})
}
}
}
}
@@ -0,0 +1,255 @@
package com.droiddeck.launcher.handtracking
import com.droiddeck.launcher.glasses.Vec3
import kotlin.math.abs
import kotlin.math.atan2
import kotlin.math.hypot
internal class Landmark(val x: Float, val y: Float, val visibility: Float)
internal class WorldPoint(val x: Float, val y: Float, val z: Float)
internal class Intrinsics(val fx: Float, val fy: Float, val cx: Float, val cy: Float, val width: Int, val height: Int)
internal class CameraLevel(up: Vec3) {
val right: Vec3
val up: Vec3
val back: Vec3
val valid: Boolean
init {
val u = up.normalized()
val forward = Vec3(0f, 0f, 1f)
val along = u.x * forward.x + u.y * forward.y + u.z * forward.z
val horizontal = Vec3(forward.x - u.x * along, forward.y - u.y * along, forward.z - u.z * along)
valid = u.length() > 0.5f && horizontal.length() > MIN_HORIZONTAL
this.up = u
back = (horizontal * -1f).normalized()
right = this.up.cross(back)
}
fun level(v: Vec3) = Vec3(dot(v, right), dot(v, up), dot(v, back))
private fun dot(a: Vec3, b: Vec3) = a.x * b.x + a.y * b.y + a.z * b.z
companion object {
private const val MIN_HORIZONTAL = 0.25f
}
}
internal class BodyFrame(
val hands: Array<Vec3?>,
val faceYaw: Float?,
val distance: Float,
val shoulderSpan: Float,
val seen: Boolean,
val estimated: BooleanArray = BooleanArray(2),
val edgeRoom: Float = 1f,
val joints: Array<FloatArray?> = arrayOfNulls(2),
)
internal class HandFix(val palm: Vec3, val x: Float, val y: Float, val extent: Float, val points: FloatArray, val world: List<WorldPoint> = emptyList())
internal object Pnp {
fun translation(count: Int, k: Intrinsics, x: (Int) -> Float, y: (Int) -> Float, world: (Int) -> WorldPoint): Vec3? {
val ata = Array(3) { DoubleArray(3) }
val atb = DoubleArray(3)
for (i in 0 until count) {
val a = ((x(i) - k.cx) / k.fx).toDouble()
val b = ((y(i) - k.cy) / k.fy).toDouble()
val w = world(i)
add(ata, atb, doubleArrayOf(1.0, 0.0, -a), a * w.z - w.x)
add(ata, atb, doubleArrayOf(0.0, 1.0, -b), b * w.z - w.y)
}
val t = solve3(ata, atb) ?: return null
return Vec3(t[0].toFloat(), t[1].toFloat(), t[2].toFloat())
}
private fun add(ata: Array<DoubleArray>, atb: DoubleArray, row: DoubleArray, rhs: Double) {
for (i in 0..2) {
atb[i] += row[i] * rhs
for (j in 0..2) ata[i][j] += row[i] * row[j]
}
}
private fun solve3(a: Array<DoubleArray>, b: DoubleArray): DoubleArray? {
val det = det3(a)
if (abs(det) < 1e-12) return null
return DoubleArray(3) { col -> det3(Array(3) { r -> DoubleArray(3) { c -> if (c == col) b[r] else a[r][c] } }) / det }
}
private fun det3(m: Array<DoubleArray>) = m[0][0] * (m[1][1] * m[2][2] - m[1][2] * m[2][1]) -
m[0][1] * (m[1][0] * m[2][2] - m[1][2] * m[2][0]) + m[0][2] * (m[1][0] * m[2][1] - m[1][1] * m[2][0])
}
internal object BodyPose {
const val NOSE = 0
const val LEFT_EYE = 2
const val RIGHT_EYE = 5
const val LEFT_EAR = 7
const val RIGHT_EAR = 8
const val RIGHT_MOUTH = 10
const val LEFT_SHOULDER = 11
const val RIGHT_SHOULDER = 12
const val LEFT_ELBOW = 13
const val RIGHT_ELBOW = 14
const val LEFT_WRIST = 15
const val RIGHT_WRIST = 16
const val LEFT_PINKY = 17
const val RIGHT_PINKY = 18
const val LEFT_INDEX = 19
const val RIGHT_INDEX = 20
const val LEFT_HIP = 23
const val RIGHT_HIP = 24
private val ANCHORS = intArrayOf(NOSE, LEFT_EYE, RIGHT_EYE, LEFT_EAR, RIGHT_EAR, LEFT_SHOULDER, RIGHT_SHOULDER, LEFT_HIP, RIGHT_HIP)
private val WRISTS = intArrayOf(LEFT_WRIST, RIGHT_WRIST)
private val PINKIES = intArrayOf(LEFT_PINKY, RIGHT_PINKY)
private val INDEXES = intArrayOf(LEFT_INDEX, RIGHT_INDEX)
const val MIN_VISIBILITY = 0.6f
fun solve(
image: List<Landmark>,
world: List<WorldPoint>,
k: Intrinsics,
level: CameraLevel,
fixes: Array<HandFix?>? = null,
held: FloatArray? = null,
): BodyFrame? {
if (image.size <= RIGHT_HIP || world.size <= RIGHT_HIP || !level.valid) return null
val t = translation(image, world, k) ?: return null
fun at(i: Int): Vec3 {
val w = world[i]
val depth = w.z + t.z
val px = image[i].x * k.width
val py = image[i].y * k.height
return Vec3((px - k.cx) / k.fx * depth, (py - k.cy) / k.fy * depth, depth)
}
fun visible(i: Int) = image[i].visibility >= MIN_VISIBILITY && image[i].x in 0f..1f && image[i].y in 0f..1f
val eyesSeen = visible(LEFT_EYE) && visible(RIGHT_EYE)
val shouldersSeen = visible(LEFT_SHOULDER) && visible(RIGHT_SHOULDER)
val hands = arrayOfNulls<Vec3>(2)
val estimated = BooleanArray(2)
val joints = arrayOfNulls<FloatArray>(2)
if (fixes == null) {
val head = if (eyesSeen) (at(LEFT_EYE) + at(RIGHT_EYE)) * 0.5f else null
if (head != null) for (side in 0..1) {
if (!visible(WRISTS[side])) continue
val grip = (at(WRISTS[side]) + at(PINKIES[side]) + at(INDEXES[side])) * (1f / 3f)
hands[side] = level.level(grip - head)
}
} else {
val head = if (eyesSeen) eyes(image, world, k) else null
if (head != null) for (side in 0..1) {
val fix = fixes.getOrNull(side)
if (fix != null) {
hands[side] = level.level(fix.palm - head)
joints[side] = handJoints(fix.world, level)
continue
}
val depth = held?.getOrNull(side)?.takeIf { it.isFinite() && it > 0f } ?: continue
if (!visible(WRISTS[side])) continue
val gx = (image[WRISTS[side]].x + image[PINKIES[side]].x + image[INDEXES[side]].x) / 3f * k.width
val gy = (image[WRISTS[side]].y + image[PINKIES[side]].y + image[INDEXES[side]].y) / 3f * k.height
hands[side] = level.level(Vec3((gx - k.cx) / k.fx * depth, (gy - k.cy) / k.fy * depth, depth) - head)
estimated[side] = true
}
}
val faceYaw = if (visible(NOSE) && visible(LEFT_EAR) && visible(RIGHT_EAR)) {
val n = world[NOSE]
val l = world[LEFT_EAR]
val r = world[RIGHT_EAR]
val f = level.level(Vec3(n.x - (l.x + r.x) / 2, n.y - (l.y + r.y) / 2, n.z - (l.z + r.z) / 2))
if (hypot(f.x, f.z) > 0.02f) atan2(-f.x, -f.z) else null
} else null
val span = if (shouldersSeen) hypot(image[LEFT_SHOULDER].x - image[RIGHT_SHOULDER].x,
(image[LEFT_SHOULDER].y - image[RIGHT_SHOULDER].y) * k.height / k.width) else 0f
var room = 1f
for (i in EDGE_JOINTS) {
if (image[i].visibility < MIN_VISIBILITY) continue
room = minOf(room, image[i].x, 1f - image[i].x, image[i].y, 1f - image[i].y)
}
return BodyFrame(hands, faceYaw, t.z, span, eyesSeen && shouldersSeen, estimated, room, joints)
}
fun handJoints(world: List<WorldPoint>, level: CameraLevel): FloatArray? {
if (world.size < HAND_POINTS) return null
var mx = 0f
var my = 0f
var mz = 0f
for (i in PALM) {
mx += world[i].x
my += world[i].y
mz += world[i].z
}
val n = PALM.size.toFloat()
val out = FloatArray(HAND_POINTS * 3)
for (i in 0 until HAND_POINTS) {
val v = level.level(Vec3(world[i].x - mx / n, world[i].y - my / n, world[i].z - mz / n))
out[i * 3] = v.x
out[i * 3 + 1] = v.y
out[i * 3 + 2] = v.z
}
return out
}
fun eyes(image: List<Landmark>, world: List<WorldPoint>, k: Intrinsics): Vec3? {
val used = HEAD.filter { image[it].visibility >= MIN_VISIBILITY }
if (used.size < MIN_HEAD_POINTS) return null
val t = Pnp.translation(used.size, k, { image[used[it]].x * k.width }, { image[used[it]].y * k.height }, { world[used[it]] })
?: return null
if (!(t.z > MIN_DISTANCE && t.z < MAX_DISTANCE)) return null
val l = world[LEFT_EYE]
val r = world[RIGHT_EYE]
return Vec3(t.x + (l.x + r.x) / 2, t.y + (l.y + r.y) / 2, t.z + (l.z + r.z) / 2)
}
fun handFix(points: FloatArray, world: List<WorldPoint>, k: Intrinsics): HandFix? {
if (world.size < HAND_POINTS || points.size < HAND_POINTS * 2) return null
val t = Pnp.translation(HAND_POINTS, k, { points[it * 2] }, { points[it * 2 + 1] }, { world[it] }) ?: return null
if (!(t.z > MIN_HAND_DISTANCE && t.z < MAX_DISTANCE)) return null
var px = 0f
var py = 0f
var wx = 0f
var wy = 0f
var wz = 0f
for (i in PALM) {
px += points[i * 2]
py += points[i * 2 + 1]
wx += world[i].x
wy += world[i].y
wz += world[i].z
}
val n = PALM.size.toFloat()
var minX = Float.MAX_VALUE
var maxX = -Float.MAX_VALUE
var minY = Float.MAX_VALUE
var maxY = -Float.MAX_VALUE
for (i in 0 until HAND_POINTS) {
minX = minOf(minX, points[i * 2])
maxX = maxOf(maxX, points[i * 2])
minY = minOf(minY, points[i * 2 + 1])
maxY = maxOf(maxY, points[i * 2 + 1])
}
return HandFix(Vec3(t.x + wx / n, t.y + wy / n, t.z + wz / n), px / n, py / n, maxOf(maxX - minX, maxY - minY), points, world)
}
private fun translation(image: List<Landmark>, world: List<WorldPoint>, k: Intrinsics): Vec3? {
val used = ANCHORS.filter { image[it].visibility >= MIN_VISIBILITY }
if (used.size < MIN_ANCHORS) return null
val t = Pnp.translation(used.size, k, { image[used[it]].x * k.width }, { image[used[it]].y * k.height }, { world[used[it]] })
?: return null
if (!(t.z > MIN_DISTANCE && t.z < MAX_DISTANCE)) return null
return t
}
const val HAND_POINTS = 21
private val HEAD = (NOSE..RIGHT_MOUTH).toList()
private val PALM = intArrayOf(0, 5, 9, 13, 17)
private val EDGE_JOINTS = intArrayOf(LEFT_ELBOW, RIGHT_ELBOW, LEFT_WRIST, RIGHT_WRIST)
private const val MIN_ANCHORS = 4
private const val MIN_HEAD_POINTS = 4
private const val MIN_DISTANCE = 0.2f
private const val MIN_HAND_DISTANCE = 0.05f
private const val MAX_DISTANCE = 6f
}
@@ -0,0 +1,323 @@
package com.droiddeck.launcher.handtracking
import android.content.Context
import android.graphics.Bitmap
import android.graphics.Canvas
import android.graphics.Matrix
import android.hardware.Sensor
import android.hardware.SensorEvent
import android.hardware.SensorEventListener
import android.hardware.SensorManager
import android.hardware.camera2.CaptureRequest
import android.os.Build
import android.os.PowerManager
import android.os.SystemClock
import android.util.Log
import android.util.Range
import android.util.Size
import androidx.camera.camera2.interop.Camera2CameraInfo
import androidx.camera.camera2.interop.Camera2Interop
import androidx.camera.camera2.interop.ExperimentalCamera2Interop
import androidx.camera.core.CameraSelector
import androidx.camera.core.ImageAnalysis
import androidx.camera.core.ImageProxy
import androidx.camera.core.resolutionselector.AspectRatioStrategy
import androidx.camera.core.resolutionselector.ResolutionSelector
import androidx.camera.core.resolutionselector.ResolutionStrategy
import androidx.camera.lifecycle.ProcessCameraProvider
import androidx.core.content.ContextCompat
import androidx.lifecycle.Lifecycle
import androidx.lifecycle.LifecycleOwner
import androidx.lifecycle.LifecycleRegistry
import com.droiddeck.launcher.glasses.Vec3
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.core.Delegate
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.poselandmarker.PoseLandmarker
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import java.io.File
import java.nio.ByteBuffer
import java.nio.ByteOrder
import java.util.concurrent.Executors
import kotlin.math.abs
@androidx.annotation.OptIn(ExperimentalCamera2Interop::class)
internal class HandCamera(
context: Context,
private val output: File?,
private val preview: Boolean,
) : LifecycleOwner {
class View(
val image: Bitmap,
val front: Boolean,
val landmarks: List<Landmark>?,
val placement: Placement,
val advice: Advice,
)
private val context = context.applicationContext
private val registry = LifecycleRegistry(this)
override val lifecycle: Lifecycle get() = registry
private val executor = Executors.newSingleThreadExecutor { Thread(it, "hand-camera") }
private val sensors = this.context.getSystemService(SensorManager::class.java)
private val power = this.context.getSystemService(PowerManager::class.java)
private val state = MutableStateFlow<View?>(null)
val view: StateFlow<View?> = state.asStateFlow()
private val broken = MutableStateFlow(false)
val failed: StateFlow<Boolean> = broken.asStateFlow()
@Volatile private var up: Vec3? = null
@Volatile private var thermal = 0
@Volatile private var stopped = false
@Volatile var bareHands = false
set(value) {
if (field != value) Log.i(TAG, if (value) "no controllers: tracking bare hands" else "controllers in use: placing them only")
field = value
}
private var provider: ProcessCameraProvider? = null
private var analysis: ImageAnalysis? = null
private var landmarker: PoseLandmarker? = null
private var hands: HandModel? = null
private val heldDepth = FloatArray(2) { Float.NaN }
private val heldAt = LongArray(2)
private var shm: HandCameraShm? = null
private var camera: TrackingCamera? = null
private var zoom = 1f
private var lastRun = 0L
private var lastStampMs = 0L
private var skipped = 0
private var recorder: HandCameraRecorder? = null
private var rawFrame: Bitmap? = null
private var turnedFrame: Bitmap? = null
private val gravity = object : SensorEventListener {
override fun onSensorChanged(event: SensorEvent) {
val v = Vec3(event.values[0], event.values[1], event.values[2])
val last = up
up = if (event.sensor.type == Sensor.TYPE_GRAVITY || last == null) v else last + (v - last) * ACCEL_SMOOTHING
}
override fun onAccuracyChanged(sensor: Sensor, accuracy: Int) {}
}
private val heat: Any? = if (Build.VERSION.SDK_INT >= 29) PowerManager.OnThermalStatusChangedListener { thermal = it } else null
init {
registry.currentState = Lifecycle.State.CREATED
}
fun start(side: String?) {
val chosen = TrackingCameras.pick(HandTracking.cameras(context), side)
if (chosen == null) {
broken.value = true
return
}
camera = chosen
zoom = chosen.zoom()
recorder = HandCameraRecorder.start(context, output, preview, chosen)
val sensor = sensors?.getDefaultSensor(Sensor.TYPE_GRAVITY) ?: sensors?.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)
if (sensor != null) sensors?.registerListener(gravity, sensor, SensorManager.SENSOR_DELAY_UI)
if (Build.VERSION.SDK_INT >= 29) {
thermal = power?.currentThermalStatus ?: 0
power?.addThermalStatusListener(ContextCompat.getMainExecutor(context), heat as PowerManager.OnThermalStatusChangedListener)
}
executor.execute { prepare() }
val future = ProcessCameraProvider.getInstance(context)
future.addListener({
if (stopped) return@addListener
runCatching { bind(future.get(), chosen) }.onFailure {
Log.w(TAG, "camera ${chosen.id} could not start", it)
broken.value = true
}
}, ContextCompat.getMainExecutor(context))
}
fun stop() {
if (stopped) return
stopped = true
sensors?.unregisterListener(gravity)
if (Build.VERSION.SDK_INT >= 29) power?.removeThermalStatusListener(heat as PowerManager.OnThermalStatusChangedListener)
analysis?.let {
it.clearAnalyzer()
runCatching { provider?.unbind(it) }
}
registry.currentState = Lifecycle.State.DESTROYED
executor.execute {
runCatching { landmarker?.close() }
landmarker = null
hands?.close()
hands = null
shm?.close()
shm = null
recorder?.close()
recorder = null
}
executor.shutdown()
}
private fun bind(cameras: ProcessCameraProvider, chosen: TrackingCamera) {
val selector = CameraSelector.Builder()
.addCameraFilter { infos -> infos.filter { Camera2CameraInfo.from(it).cameraId == chosen.id } }
.build()
val builder = ImageAnalysis.Builder()
val extender = Camera2Interop.Extender(builder)
chosen.fps?.let { extender.setCaptureRequestOption(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE, Range(it.first, it.second)) }
recorder?.let { extender.setSessionCaptureCallback(it.callback) }
val use = builder
.setResolutionSelector(
ResolutionSelector.Builder()
.setAspectRatioStrategy(AspectRatioStrategy.RATIO_4_3_FALLBACK_AUTO_STRATEGY)
.setResolutionStrategy(ResolutionStrategy(Size(ANALYSIS_WIDTH, ANALYSIS_HEIGHT), ResolutionStrategy.FALLBACK_RULE_CLOSEST_HIGHER_THEN_LOWER))
.build(),
)
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
.build()
use.setAnalyzer(executor, ::analyze)
val bound = cameras.bindToLifecycle(this, selector, use)
if (zoom < 1f) bound.cameraControl.setZoomRatio(zoom)
provider = cameras
analysis = use
registry.currentState = Lifecycle.State.RESUMED
Log.i(TAG, "camera ${chosen.id} (${if (chosen.front) "front" else "rear"}, ${chosen.fovDegrees.toInt()}°, zoom $zoom, ${chosen.fps} fps) " +
if (preview) "checking placement" else "tracking hands")
}
private fun prepare() {
if (stopped) return
runCatching {
val bytes = context.assets.open(MODEL).use { it.readBytes() }
val model = ByteBuffer.allocateDirect(bytes.size).order(ByteOrder.nativeOrder()).put(bytes)
model.rewind()
val options = PoseLandmarker.PoseLandmarkerOptions.builder()
.setBaseOptions(BaseOptions.builder().setModelAssetBuffer(model).setDelegate(Delegate.CPU).build())
.setRunningMode(RunningMode.VIDEO)
.setNumPoses(1)
.setMinPoseDetectionConfidence(MIN_CONFIDENCE)
.setMinPosePresenceConfidence(MIN_CONFIDENCE)
.setOutputSegmentationMasks(false)
.build()
landmarker = PoseLandmarker.createFromOptions(context, options)
if (output != null) shm = HandCameraShm.open(output)
}.onFailure {
Log.w(TAG, "pose model could not load", it)
broken.value = true
}
if (landmarker != null) hands = runCatching { HandModel.open(context) }.onFailure { Log.w(TAG, "hand model could not load", it) }.getOrNull()
}
private fun analyze(image: ImageProxy) {
try {
val chosen = camera ?: return
val pose = landmarker ?: return
val arrival = System.nanoTime()
if (stopped || arrival - lastRun < interval()) {
skipped++
return
}
lastRun = arrival
val exposed = exposure(image.imageInfo.timestamp, arrival, chosen)
val stampMs = maxOf(exposed / 1_000_000L, lastStampMs + 1)
lastStampMs = stampMs
val gravity = up
val placement = if (gravity != null) CameraGeometry.placement(chosen.front, gravity) else Placement.UNKNOWN
val turns = if (gravity != null) CameraGeometry.uprightTurns(chosen.front, chosen.orientation, gravity)
else image.imageInfo.rotationDegrees / 90
val frame = upright(raw(image), turns)
val result = pose.detectForVideo(BitmapImageBuilder(frame).build(), stampMs)
val inferred = System.nanoTime()
val found = result.landmarks().firstOrNull()
val marks = found?.map { Landmark(it.x(), it.y(), it.visibility().orElse(0f)) }
val world = result.worldLandmarks().firstOrNull()?.map { WorldPoint(it.x(), it.y(), it.z()) }
val k = chosen.intrinsics(image.width, image.height, zoom, turns)
val model = hands
val fixes = if (marks != null && model != null) model.find(frame, marks, k, stampMs) else null
val handsDone = System.nanoTime()
fixes?.forEachIndexed { side, fix ->
if (fix != null) {
heldDepth[side] = fix.palm.z
heldAt[side] = exposed
}
}
val held = FloatArray(2) { if (exposed - heldAt[it] <= HOLD_NS) heldDepth[it] else Float.NaN }
val body = if (marks != null && world != null && gravity != null) {
BodyPose.solve(marks, world, k, CameraLevel(CameraGeometry.upInCamera(chosen.front, chosen.orientation, turns, gravity)),
if (model != null) fixes ?: arrayOfNulls(2) else null, held)
} else null
shm?.write(exposed, body, bareHands)
if (preview) state.value = View(frame, chosen.front, marks, placement, PlacementAdvice.of(placement, body))
recorder?.frame(HandCameraRecorder.Frame(exposed, image.imageInfo.timestamp, arrival, inferred, System.nanoTime(), skipped,
thermal, turns, placement, gravity, k,
found?.map { floatArrayOf(it.x(), it.y(), it.z(), it.visibility().orElse(0f), it.presence().orElse(0f)) },
world, body, handsDone, fixes), frame)
skipped = 0
} catch (e: Exception) {
Log.w(TAG, "camera frame skipped", e)
} finally {
image.close()
}
}
private fun raw(image: ImageProxy): Bitmap {
val plane = image.planes[0]
if (plane.pixelStride != 4 || plane.rowStride != image.width * 4) return image.toBitmap()
val bitmap = rawFrame?.takeIf { it.width == image.width && it.height == image.height }
?: Bitmap.createBitmap(image.width, image.height, Bitmap.Config.ARGB_8888).also { rawFrame = it }
plane.buffer.rewind()
bitmap.copyPixelsFromBuffer(plane.buffer)
return bitmap
}
private fun upright(raw: Bitmap, turns: Int): Bitmap {
val quarter = Math.floorMod(turns, 4)
if (quarter == 0) return if (preview) raw.copy(Bitmap.Config.ARGB_8888, false) else raw
val width = if (quarter % 2 == 1) raw.height else raw.width
val height = if (quarter % 2 == 1) raw.width else raw.height
val target = if (preview) Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
else turnedFrame?.takeIf { it.width == width && it.height == height }
?: Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888).also { turnedFrame = it }
val matrix = Matrix()
matrix.postRotate(90f * quarter)
when (quarter) {
1 -> matrix.postTranslate(raw.height.toFloat(), 0f)
2 -> matrix.postTranslate(raw.width.toFloat(), raw.height.toFloat())
else -> matrix.postTranslate(0f, raw.width.toFloat())
}
Canvas(target).drawBitmap(raw, matrix, null)
return target
}
private fun exposure(stamp: Long, arrival: Long, chosen: TrackingCamera): Long {
val monotonic = if (chosen.realtime) stamp - (SystemClock.elapsedRealtimeNanos() - System.nanoTime()) else stamp
return if (abs(arrival - monotonic) < MAX_DELAY_NS) monotonic else arrival - TYPICAL_DELAY_NS
}
private fun interval(): Long {
val fps = when {
thermal >= SEVERE -> HOT_FPS
thermal >= MODERATE -> WARM_FPS
else -> FPS
}
return 1_000_000_000L / fps - SLACK_NS
}
companion object {
private const val TAG = "HandCamera"
private const val MODEL = "handtracking/pose_landmarker_lite.task"
private const val ANALYSIS_WIDTH = 960
private const val ANALYSIS_HEIGHT = 720
private const val MIN_CONFIDENCE = 0.5f
private const val FPS = 15
private const val WARM_FPS = 10
private const val HOT_FPS = 6
private const val MODERATE = 2
private const val SEVERE = 3
private const val SLACK_NS = 8_000_000L
private const val MAX_DELAY_NS = 1_000_000_000L
private const val TYPICAL_DELAY_NS = 60_000_000L
private const val ACCEL_SMOOTHING = 0.1f
private const val HOLD_NS = 500_000_000L
}
}
@@ -0,0 +1,203 @@
package com.droiddeck.launcher.handtracking
import android.content.Context
import android.graphics.Bitmap
import android.hardware.camera2.CameraCaptureSession
import android.hardware.camera2.CaptureRequest
import android.hardware.camera2.CaptureResult
import android.hardware.camera2.TotalCaptureResult
import android.util.Log
import com.droiddeck.launcher.glasses.Vec3
import java.io.BufferedWriter
import java.io.File
import java.io.FileOutputStream
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
import java.util.concurrent.Executors
import java.util.concurrent.atomic.AtomicBoolean
import kotlin.math.max
import kotlin.math.roundToInt
internal class HandCameraRecorder private constructor(private val dir: File, private val traceFlag: File?) {
class Capture(val exposureNs: Long, val frameDurationNs: Long, val iso: Int)
class Frame(
val exposureNs: Long,
val sensorNs: Long,
val arrivalNs: Long,
val inferredNs: Long,
val doneNs: Long,
val skipped: Int,
val thermal: Int,
val turns: Int,
val placement: Placement,
val gravity: Vec3?,
val intrinsics: Intrinsics?,
val marks: List<FloatArray>?,
val world: List<WorldPoint>?,
val body: BodyFrame?,
val handsNs: Long = 0L,
val fixes: Array<HandFix?>? = null,
)
private val writer = Executors.newSingleThreadExecutor { Thread(it, "hand-camera-record") }
private val lines: BufferedWriter = File(dir, "frames.jsonl").bufferedWriter()
private val busy = AtomicBoolean(false)
private val captures = object : LinkedHashMap<Long, Capture>() {
override fun removeEldestEntry(eldest: MutableMap.MutableEntry<Long, Capture>?) = size > KEEP_CAPTURES
}
private var count = 0
@Volatile private var closed = false
val callback = object : CameraCaptureSession.CaptureCallback() {
override fun onCaptureCompleted(session: CameraCaptureSession, request: CaptureRequest, result: TotalCaptureResult) {
val stamp = result.get(CaptureResult.SENSOR_TIMESTAMP) ?: return
val capture = Capture(result.get(CaptureResult.SENSOR_EXPOSURE_TIME) ?: 0L,
result.get(CaptureResult.SENSOR_FRAME_DURATION) ?: 0L, result.get(CaptureResult.SENSOR_SENSITIVITY) ?: 0)
synchronized(captures) { captures[stamp] = capture }
}
}
fun frame(frame: Frame, image: Bitmap) {
if (closed || count >= MAX_FRAMES) return
val index = ++count
val capture = synchronized(captures) { captures[frame.sensorNs] }
val small = if (busy.compareAndSet(false, true)) runCatching { shrink(image) }.getOrNull().also { if (it == null) busy.set(false) } else null
val name = if (small != null) "%05d.jpg".format(Locale.ROOT, index) else null
val line = json(index, frame, capture, name)
val queued = runCatching { writer.execute {
if (small != null) {
runCatching { FileOutputStream(File(dir, name!!)).use { small.compress(Bitmap.CompressFormat.JPEG, QUALITY, it) } }
small.recycle()
busy.set(false)
}
runCatching {
lines.write(line)
lines.newLine()
if (index % FLUSH_EVERY == 0) lines.flush()
}
} }
if (queued.isFailure) {
small?.recycle()
busy.set(false)
}
}
fun close() {
if (closed) return
closed = true
traceFlag?.delete()
writer.execute { runCatching { lines.close() } }
writer.shutdown()
Log.i(TAG, "recorded $count frames to $dir")
}
private fun shrink(image: Bitmap): Bitmap {
val scale = SIDE.toFloat() / max(image.width, image.height)
if (scale >= 1f) return image.copy(Bitmap.Config.ARGB_8888, false)
return Bitmap.createScaledBitmap(image, (image.width * scale).roundToInt(), (image.height * scale).roundToInt(), true)
}
private fun json(index: Int, f: Frame, capture: Capture?, jpg: String?): String = buildString {
append("{\"i\":").append(index)
append(",\"exp\":").append(f.exposureNs)
append(",\"sensor\":").append(f.sensorNs)
append(",\"arr\":").append(f.arrivalNs)
append(",\"inf\":").append(f.inferredNs)
append(",\"done\":").append(f.doneNs)
append(",\"hand\":").append(f.handsNs)
append(",\"skip\":").append(f.skipped)
append(",\"heat\":").append(f.thermal)
append(",\"turns\":").append(f.turns)
append(",\"place\":\"").append(f.placement.name).append('"')
append(",\"g\":").vec(f.gravity)
f.intrinsics?.let { append(",\"k\":[").num(it.fx).append(',').num(it.cx).append(',').num(it.cy).append(',').append(it.width).append(',').append(it.height).append(']') }
capture?.let { append(",\"ae\":[").append(it.exposureNs).append(',').append(it.frameDurationNs).append(',').append(it.iso).append(']') }
append(",\"lm\":")
if (f.marks == null) append("null") else {
append('[')
f.marks.forEachIndexed { i, m ->
if (i > 0) append(',')
append('[')
m.forEachIndexed { j, v -> if (j > 0) append(','); num(v) }
append(']')
}
append(']')
}
append(",\"wl\":")
if (f.world == null) append("null") else {
append('[')
f.world.forEachIndexed { i, w ->
if (i > 0) append(',')
append('[').num(w.x).append(',').num(w.y).append(',').num(w.z).append(']')
}
append(']')
}
f.body?.let { b ->
append(",\"hands\":[").vec(b.hands[0]).append(',').vec(b.hands[1]).append(']')
append(",\"face\":"); if (b.faceYaw == null) append("null") else num(b.faceYaw)
append(",\"est\":[").append(b.estimated[0]).append(',').append(b.estimated[1]).append(']')
append(",\"room\":").num(b.edgeRoom)
append(",\"dist\":").num(b.distance)
append(",\"span\":").num(b.shoulderSpan)
append(",\"joints\":[")
b.joints.forEachIndexed { i, j ->
if (i > 0) append(',')
if (j == null) append("null") else {
append('[')
j.forEachIndexed { k, v -> if (k > 0) append(','); num(v) }
append(']')
}
}
append(']')
}
f.fixes?.let { fixes ->
append(",\"fix\":[")
fixes.forEachIndexed { i, fix ->
if (i > 0) append(',')
if (fix == null) append("null") else {
append("{\"palm\":").vec(fix.palm).append(",\"at\":[").num(fix.x).append(',').num(fix.y).append("],\"size\":").num(fix.extent)
append(",\"pts\":[")
fix.points.forEachIndexed { j, v -> if (j > 0) append(','); num(v) }
append("],\"world\":[")
fix.world.forEachIndexed { j, w -> if (j > 0) append(','); num(w.x).append(',').num(w.y).append(',').num(w.z) }
append("]}")
}
}
append(']')
}
append(",\"jpg\":"); if (jpg == null) append("null") else append('"').append(jpg).append('"')
append('}')
}
private fun StringBuilder.num(v: Float): StringBuilder = if (v.isFinite()) append(v) else append("null")
private fun StringBuilder.vec(v: Vec3?): StringBuilder =
if (v == null) append("null") else append('[').num(v.x).append(',').num(v.y).append(',').num(v.z).append(']')
companion object {
private const val TAG = "HandCamera"
const val FLAG = "handcam-record"
const val TRACE_FLAG = "droiddeck-handtrace.on"
private const val MAX_FRAMES = 5400
private const val KEEP_CAPTURES = 32
private const val SIDE = 640
private const val QUALITY = 85
private const val FLUSH_EVERY = 30
fun start(context: Context, output: File?, preview: Boolean, camera: TrackingCamera): HandCameraRecorder? {
if (!File(context.filesDir, FLAG).exists()) return null
return runCatching {
val stamp = SimpleDateFormat("yyyyMMdd-HHmmss", Locale.ROOT).format(Date())
val dir = File(context.cacheDir, "handcam-rec/$stamp-${if (preview) "check" else "track"}").apply { mkdirs() }
File(dir, "camera.json").writeText("{\"id\":\"${camera.id}\",\"front\":${camera.front},\"orientation\":${camera.orientation}," +
"\"focal\":${camera.focalPixels},\"active\":[${camera.activeWidth},${camera.activeHeight}]," +
"\"fps\":${camera.fps?.let { "[${it.first},${it.second}]" } ?: "null"}}")
val flag = output?.parentFile?.let { File(it, TRACE_FLAG) }?.also { it.createNewFile() }
Log.i(TAG, "recording hand camera to $dir")
HandCameraRecorder(dir, flag)
}.onFailure { Log.w(TAG, "hand camera recording unavailable", it) }.getOrNull()
}
}
}
@@ -0,0 +1,103 @@
package com.droiddeck.launcher.handtracking
import android.util.Log
import com.droiddeck.launcher.glasses.Vec3
import com.droiddeck.launcher.input.FakeInputWriter
import java.io.File
import java.io.RandomAccessFile
import java.nio.ByteOrder
import java.nio.MappedByteBuffer
import java.nio.channels.FileChannel
internal class HandCameraShm private constructor(
private val file: RandomAccessFile,
private val map: MappedByteBuffer,
private val fence: () -> Unit,
) {
private var seq = map.getInt(SEQ).let { if (it and 1 != 0) it + 1 else it }
private var frame = 0
init {
begin()
map.putInt(MAGIC_AT, MAGIC)
map.putInt(VERSION_AT, VERSION)
for (at in FLAGS until SIZE step 4) map.putInt(at, 0)
end()
}
fun write(timeNs: Long, body: BodyFrame?, bareHands: Boolean = false) {
begin()
val face = body?.faceYaw
map.putInt(FLAGS, RUNNING or (if (face != null) FACE else 0) or (if (bareHands) BARE_HANDS else 0))
map.putLong(TIME, timeNs)
map.putInt(FRAME, ++frame)
map.putFloat(FACE_YAW, face ?: 0f)
for (side in 0..1) {
val at = HANDS + side * HAND_SIZE
val hand: Vec3? = body?.hands?.getOrNull(side)
val joints = if (hand != null) body?.joints?.getOrNull(side)?.takeIf { it.size == JOINT_FLOATS } else null
val seen = if (body?.estimated?.getOrNull(side) == true) SEEN or ESTIMATED else SEEN
map.putInt(at, if (hand == null) 0 else if (joints != null) seen or JOINTS else seen)
map.putFloat(at + 4, hand?.x ?: 0f)
map.putFloat(at + 8, hand?.y ?: 0f)
map.putFloat(at + 12, hand?.z ?: 0f)
for (i in 0 until JOINT_FLOATS) map.putFloat(at + JOINTS_AT + i * 4, joints?.get(i) ?: 0f)
}
end()
}
fun close() {
begin()
for (at in FLAGS until SIZE step 4) map.putInt(at, 0)
end()
runCatching { file.close() }
}
private fun begin() {
map.putInt(SEQ, ++seq)
fence()
}
private fun end() {
fence()
map.putInt(SEQ, ++seq)
}
companion object {
const val FILE = "droiddeck-handcam"
const val SIZE = 568
private const val MAGIC = 0x4D434844
private const val VERSION = 2
private const val MAGIC_AT = 0
private const val VERSION_AT = 4
private const val SEQ = 8
private const val FLAGS = 12
private const val TIME = 16
private const val FRAME = 24
private const val FACE_YAW = 28
private const val HANDS = 32
private const val HAND_SIZE = 268
private const val JOINTS_AT = 16
private const val JOINT_FLOATS = BodyPose.HAND_POINTS * 3
private const val RUNNING = 1
private const val FACE = 2
private const val BARE_HANDS = 4
private const val SEEN = 1
private const val ESTIMATED = 2
private const val JOINTS = 4
fun open(path: File, fence: () -> Unit = FakeInputWriter::storeFence): HandCameraShm? = runCatching {
path.parentFile?.mkdirs()
val file = RandomAccessFile(path, "rw")
try {
file.setLength(SIZE.toLong())
val map = file.channel.map(FileChannel.MapMode.READ_WRITE, 0, SIZE.toLong())
map.order(ByteOrder.LITTLE_ENDIAN)
HandCameraShm(file, map, fence)
} catch (e: Exception) {
file.close()
throw e
}
}.onFailure { Log.w("HandCamera", "hand camera file unavailable: $path", it) }.getOrNull()
}
}
@@ -0,0 +1,172 @@
package com.droiddeck.launcher.handtracking
import android.content.Context
import android.graphics.Bitmap
import android.graphics.Canvas
import android.graphics.Color
import android.graphics.Matrix
import android.graphics.Paint
import com.google.mediapipe.framework.image.BitmapImageBuilder
import com.google.mediapipe.tasks.components.containers.Landmark as MpLandmark
import com.google.mediapipe.tasks.components.containers.NormalizedLandmark
import com.google.mediapipe.tasks.core.BaseOptions
import com.google.mediapipe.tasks.core.Delegate
import com.google.mediapipe.tasks.vision.core.RunningMode
import com.google.mediapipe.tasks.vision.handlandmarker.HandLandmarker
import java.nio.ByteBuffer
import java.nio.ByteOrder
import kotlin.math.hypot
import kotlin.math.max
internal class HandModel private constructor(private val landmarkers: List<HandLandmarker>, private val starter: HandLandmarker) {
private class Track(val x: Float, val y: Float, val size: Float)
private val crops = List(2) { Bitmap.createBitmap(CROP, CROP, Bitmap.Config.ARGB_8888) }
private val startCrop = Bitmap.createBitmap(START_CROP, START_CROP, Bitmap.Config.ARGB_8888)
private val tracks = arrayOfNulls<Track>(2)
private val starts = IntArray(2)
private val paint = Paint(Paint.FILTER_BITMAP_FLAG)
private val matrix = Matrix()
fun find(frame: Bitmap, marks: List<Landmark>, k: Intrinsics, stampMs: Long): Array<HandFix?> {
val fixes = arrayOfNulls<HandFix>(2)
if (marks.size <= BodyPose.RIGHT_INDEX) return fixes
val shoulders = hypot((marks[BodyPose.LEFT_SHOULDER].x - marks[BodyPose.RIGHT_SHOULDER].x) * k.width,
(marks[BodyPose.LEFT_SHOULDER].y - marks[BodyPose.RIGHT_SHOULDER].y) * k.height)
for (side in 0..1) {
val wrist = marks[WRISTS[side]]
val wx = wrist.x * k.width
val wy = wrist.y * k.height
val fix = tracks[side]?.let { follow(frame, it, side, k, stampMs) }?.takeIf { belongs(it, marks, side, shoulders, k) }
?: start(frame, marks, side, wx, wy, shoulders, k)?.takeIf { belongs(it, marks, side, shoulders, k) }
if (fix == null) {
tracks[side] = null
continue
}
fixes[side] = fix
starts[side] = 0
tracks[side] = Track(fix.x, fix.y, max(fix.extent, shoulders * MIN_HAND))
}
val left = fixes[0]
val right = fixes[1]
if (left != null && right != null && hypot(left.x - right.x, left.y - right.y) < max(left.extent, right.extent) * SAME_HAND) {
val drop = if (gap(left, marks, 0, k) <= gap(right, marks, 1, k)) 1 else 0
fixes[drop] = null
tracks[drop] = null
}
return fixes
}
fun close() {
landmarkers.forEach { runCatching { it.close() } }
runCatching { starter.close() }
crops.forEach { it.recycle() }
startCrop.recycle()
}
private fun follow(frame: Bitmap, track: Track, side: Int, k: Intrinsics, stampMs: Long): HandFix? {
val size = max(track.size * CROP_HANDS, MIN_CROP)
val x0 = track.x - size / 2
val y0 = track.y - size / 2
draw(frame, crops[side], x0, y0, size)
val result = landmarkers[side].detectForVideo(BitmapImageBuilder(crops[side]).build(), stampMs)
val found = result.landmarks().firstOrNull() ?: return null
val world = result.worldLandmarks().firstOrNull() ?: return null
return fixOf(found, world, x0, y0, size, k)
}
private fun start(frame: Bitmap, marks: List<Landmark>, side: Int, wx: Float, wy: Float, shoulders: Float, k: Intrinsics): HandFix? {
val wrist = marks[WRISTS[side]]
if (wrist.visibility < MIN_VISIBILITY || shoulders < 1f) return null
if (wrist.x < -EDGE || wrist.x > 1f + EDGE || wrist.y < -EDGE || wrist.y > 1f + EDGE) return null
val wide = starts[side]++ % 2 == 0
val tx = (marks[PINKIES[side]].x + marks[INDEXES[side]].x) / 2 * k.width - wx
val ty = (marks[PINKIES[side]].y + marks[INDEXES[side]].y) / 2 * k.height - wy
val length = hypot(tx, ty)
val reach = if (wide) 0f else shoulders * TOWARD_FINGERS
val cx = if (length > 1e-3f) wx + tx / length * reach else wx
val cy = if (length > 1e-3f) wy + ty / length * reach else wy
val size = max(shoulders * if (wide) WIDE_SHOULDERS else START_SHOULDERS, MIN_CROP)
val x0 = cx - size / 2
val y0 = cy - size / 2
draw(frame, startCrop, x0, y0, size)
val result = starter.detect(BitmapImageBuilder(startCrop).build())
var best: HandFix? = null
var bestGap = Float.MAX_VALUE
result.landmarks().forEachIndexed { i, found ->
val world = result.worldLandmarks().getOrNull(i) ?: return@forEachIndexed
val fix = fixOf(found, world, x0, y0, size, k) ?: return@forEachIndexed
val gap = hypot(fix.points[0] - wx, fix.points[1] - wy)
if (gap < bestGap) {
best = fix
bestGap = gap
}
}
return best
}
private fun draw(frame: Bitmap, crop: Bitmap, x0: Float, y0: Float, size: Float) {
crop.eraseColor(Color.BLACK)
matrix.setTranslate(-x0, -y0)
matrix.postScale(crop.width / size, crop.height / size)
Canvas(crop).drawBitmap(frame, matrix, paint)
}
private fun fixOf(found: List<NormalizedLandmark>, world: List<MpLandmark>, x0: Float, y0: Float, size: Float, k: Intrinsics): HandFix? {
if (found.size < BodyPose.HAND_POINTS || world.size < BodyPose.HAND_POINTS) return null
val points = FloatArray(BodyPose.HAND_POINTS * 2) { i ->
val p = found[i / 2]
if (i % 2 == 0) x0 + p.x() * size else y0 + p.y() * size
}
return BodyPose.handFix(points, world.map { WorldPoint(it.x(), it.y(), it.z()) }, k)
}
private fun belongs(fix: HandFix, marks: List<Landmark>, side: Int, shoulders: Float, k: Intrinsics): Boolean {
val own = marks[WRISTS[side]]
if (hypot(fix.x - own.x * k.width, fix.y - own.y * k.height) > max(fix.extent, shoulders) * MAX_WRIST_GAP) return false
val other = marks[WRISTS[1 - side]]
if (other.visibility < MIN_VISIBILITY || other.x !in 0f..1f || other.y !in 0f..1f) return true
return hypot(fix.points[0] - other.x * k.width, fix.points[1] - other.y * k.height) >=
hypot(fix.points[0] - own.x * k.width, fix.points[1] - own.y * k.height)
}
private fun gap(fix: HandFix, marks: List<Landmark>, side: Int, k: Intrinsics): Float =
hypot(fix.x - marks[WRISTS[side]].x * k.width, fix.y - marks[WRISTS[side]].y * k.height)
companion object {
private const val MODEL = "handtracking/hand_landmarker.task"
private val WRISTS = intArrayOf(BodyPose.LEFT_WRIST, BodyPose.RIGHT_WRIST)
private val PINKIES = intArrayOf(BodyPose.LEFT_PINKY, BodyPose.RIGHT_PINKY)
private val INDEXES = intArrayOf(BodyPose.LEFT_INDEX, BodyPose.RIGHT_INDEX)
private const val CROP = 256
private const val START_CROP = 384
private const val CROP_HANDS = 2.8f
private const val MIN_CROP = 48f
private const val START_SHOULDERS = 2.5f
private const val WIDE_SHOULDERS = 4f
private const val MIN_HAND = 0.25f
private const val TOWARD_FINGERS = 0.35f
private const val MAX_WRIST_GAP = 3f
private const val SAME_HAND = 0.5f
private const val EDGE = 0.05f
private const val MIN_VISIBILITY = 0.5f
private const val MIN_CONFIDENCE = 0.4f
fun open(context: Context): HandModel {
val bytes = context.assets.open(MODEL).use { it.readBytes() }
fun landmarker(mode: RunningMode, hands: Int): HandLandmarker {
val model = ByteBuffer.allocateDirect(bytes.size).order(ByteOrder.nativeOrder()).put(bytes)
model.rewind()
return HandLandmarker.createFromOptions(context, HandLandmarker.HandLandmarkerOptions.builder()
.setBaseOptions(BaseOptions.builder().setModelAssetBuffer(model).setDelegate(Delegate.CPU).build())
.setRunningMode(mode)
.setNumHands(hands)
.setMinHandDetectionConfidence(MIN_CONFIDENCE)
.setMinHandPresenceConfidence(MIN_CONFIDENCE)
.setMinTrackingConfidence(MIN_CONFIDENCE)
.build())
}
return HandModel(List(2) { landmarker(RunningMode.VIDEO, 1) }, landmarker(RunningMode.IMAGE, 2))
}
}
}
@@ -0,0 +1,110 @@
package com.droiddeck.launcher.handtracking
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.os.Build
import com.droiddeck.launcher.input.ControllerPrefs
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
internal object HandTracking {
data class Settings(
val enabled: Boolean = false,
val asked: Boolean = false,
val side: String? = null,
val permitted: Boolean = false,
val sides: Set<String> = emptySet(),
) {
val usable: Boolean get() = sides.isNotEmpty()
val ready: Boolean get() = enabled && permitted && usable
}
private val state = MutableStateFlow(Settings())
val settings: StateFlow<Settings> = state.asStateFlow()
private val checks = MutableStateFlow(0)
val checkRequests: StateFlow<Int> = checks.asStateFlow()
@Volatile private var cameras: List<TrackingCamera>? = null
val supported: Boolean get() = Build.VERSION.SDK_INT >= 31
fun load(context: Context): Settings {
val prefs = ControllerPrefs.prefs(context)
val found = cameras(context)
val next = Settings(
enabled = prefs.getBoolean(ENABLED, false),
asked = prefs.getBoolean(ASKED, false),
side = prefs.getString(SIDE, null),
permitted = permitted(context),
sides = if (supported) TrackingCameras.sides(found) else emptySet(),
)
state.value = next
return next
}
fun cameras(context: Context): List<TrackingCamera> =
cameras ?: TrackingCameras.list(context.applicationContext).also { cameras = it }
fun camera(context: Context): TrackingCamera? = TrackingCameras.pick(cameras(context), state.value.side)
fun permitted(context: Context): Boolean =
context.checkSelfPermission(Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED
fun refreshPermission(context: Context) {
state.value = state.value.copy(permitted = permitted(context))
}
fun allow(context: Context, side: String?) {
val edit = ControllerPrefs.prefs(context).edit().putBoolean(ASKED, true).putBoolean(ENABLED, true)
if (side != null) edit.putString(SIDE, side)
edit.apply()
load(context)
}
fun decline(context: Context) {
ControllerPrefs.prefs(context).edit().putBoolean(ASKED, true).putBoolean(ENABLED, false).apply()
load(context)
}
fun setEnabled(context: Context, on: Boolean) {
ControllerPrefs.prefs(context).edit().putBoolean(ASKED, true).putBoolean(ENABLED, on).apply()
load(context)
}
fun setSide(context: Context, side: String) {
ControllerPrefs.prefs(context).edit().putString(SIDE, side).apply()
load(context)
}
fun shouldAsk(settings: Settings, headset: Boolean): Boolean = headset && !settings.asked && settings.usable
fun wanted(context: Context): Boolean = supported && load(context).ready
fun requestCheck() {
checks.value++
}
private const val ENABLED = "handCamera"
private const val ASKED = "handCameraAsked"
private const val SIDE = "handCameraSide"
}
internal enum class Advice { STARTING, STAND_UP, UPRIGHT, NOT_SEEN, MOVE_BACK, MOVE_CLOSER, SHOW_HANDS, GOOD }
internal object PlacementAdvice {
const val MIN_SPAN = 0.12f
const val MAX_SPAN = 0.26f
const val MIN_ROOM = 0.05f
fun of(placement: Placement, body: BodyFrame?): Advice = when {
placement == Placement.FACE_UP || placement == Placement.FACE_DOWN -> Advice.STAND_UP
placement == Placement.POINTS_UP || placement == Placement.POINTS_DOWN -> Advice.UPRIGHT
body == null || !body.seen -> Advice.NOT_SEEN
body.shoulderSpan > MAX_SPAN -> Advice.MOVE_BACK
body.shoulderSpan < MIN_SPAN -> Advice.MOVE_CLOSER
body.edgeRoom < MIN_ROOM -> Advice.MOVE_BACK
body.hands.any { it == null } -> Advice.SHOW_HANDS
else -> Advice.GOOD
}
}
@@ -0,0 +1,136 @@
package com.droiddeck.launcher.handtracking
import android.content.Context
import android.hardware.camera2.CameraCharacteristics
import android.hardware.camera2.CameraManager
import android.os.Build
import com.droiddeck.launcher.glasses.Vec3
import kotlin.math.asin
import kotlin.math.atan
import kotlin.math.max
import kotlin.math.min
import kotlin.math.tan
internal class TrackingCamera(
val id: String,
val front: Boolean,
val orientation: Int,
val focalPixels: Float,
val activeWidth: Int,
val activeHeight: Int,
val minZoom: Float,
val realtime: Boolean,
val fps: Pair<Int, Int>? = null,
) {
val fovDegrees: Float get() = Math.toDegrees(2.0 * atan(activeWidth / 2.0 / focalPixels)).toFloat()
fun zoom(): Float {
if (minZoom >= 1f) return 1f
val needed = (tan(Math.toRadians(fovDegrees / 2.0)) / tan(Math.toRadians(MAX_FOV / 2.0))).toFloat()
return min(1f, max(minZoom, needed))
}
fun intrinsics(bufferWidth: Int, bufferHeight: Int, zoom: Float, turns: Int): Intrinsics {
val aspect = bufferWidth.toFloat() / bufferHeight
val region = min(activeWidth.toFloat(), activeHeight * aspect) / zoom
val f = focalPixels * bufferWidth / region
val quarter = turns and 1 == 1
val w = if (quarter) bufferHeight else bufferWidth
val h = if (quarter) bufferWidth else bufferHeight
return Intrinsics(f, f, w / 2f, h / 2f, w, h)
}
companion object {
const val MAX_FOV = 100.0
}
}
internal object TrackingCameras {
const val REAR = "rear"
const val FRONT = "front"
private const val MAX_FPS = 30
fun list(context: Context): List<TrackingCamera> {
val manager = context.getSystemService(CameraManager::class.java) ?: return emptyList()
val ids = runCatching { manager.cameraIdList.toList() }.getOrDefault(emptyList())
return ids.mapNotNull { id -> runCatching { describe(id, manager.getCameraCharacteristics(id)) }.getOrNull() }
}
private fun describe(id: String, c: CameraCharacteristics): TrackingCamera? {
val facing = c.get(CameraCharacteristics.LENS_FACING) ?: return null
if (facing == CameraCharacteristics.LENS_FACING_EXTERNAL) return null
val caps = c.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES) ?: return null
if (CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_BACKWARD_COMPATIBLE !in caps) return null
val active = c.get(CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE) ?: return null
val pixels = c.get(CameraCharacteristics.SENSOR_INFO_PIXEL_ARRAY_SIZE) ?: return null
val physical = c.get(CameraCharacteristics.SENSOR_INFO_PHYSICAL_SIZE) ?: return null
val focal = c.get(CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS)?.firstOrNull() ?: return null
val calibrated = c.get(CameraCharacteristics.LENS_INTRINSIC_CALIBRATION)?.firstOrNull()?.takeIf { it > 1f }
val focalPixels = calibrated ?: (focal * pixels.width / physical.width)
val minZoom = if (Build.VERSION.SDK_INT >= 30) c.get(CameraCharacteristics.CONTROL_ZOOM_RATIO_RANGE)?.lower ?: 1f else 1f
val realtime = c.get(CameraCharacteristics.SENSOR_INFO_TIMESTAMP_SOURCE) == CameraCharacteristics.SENSOR_INFO_TIMESTAMP_SOURCE_REALTIME
val fps = steadyFps(c.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES)?.map { it.lower to it.upper }.orEmpty())
return TrackingCamera(id, facing == CameraCharacteristics.LENS_FACING_FRONT, c.get(CameraCharacteristics.SENSOR_ORIENTATION) ?: 0,
focalPixels, active.width(), active.height(), minZoom, realtime, fps)
}
fun steadyFps(ranges: List<Pair<Int, Int>>): Pair<Int, Int>? =
ranges.filter { it.second <= MAX_FPS }.maxWithOrNull(compareBy<Pair<Int, Int>> { it.first }.thenByDescending { it.second })
fun sides(cameras: List<TrackingCamera>): Set<String> = cameras.mapTo(LinkedHashSet()) { if (it.front) FRONT else REAR }
fun pick(cameras: List<TrackingCamera>, side: String?): TrackingCamera? {
val wanted = cameras.filter { side == null || (it.front == (side == FRONT)) }.ifEmpty { cameras }
val preferred = if (side == null) wanted.filter { !it.front }.ifEmpty { wanted } else wanted
return preferred.maxWithOrNull(compareBy<TrackingCamera> { min(widest(it), TrackingCamera.MAX_FOV.toFloat()) }
.thenBy { it.activeWidth.toLong() * it.activeHeight })
}
private fun widest(c: TrackingCamera): Float =
Math.toDegrees(2.0 * atan(tan(Math.toRadians(c.fovDegrees / 2.0)) / c.zoom())).toFloat()
}
internal object CameraGeometry {
fun axes(front: Boolean, orientation: Int, turns: Int): Pair<Vec3, Vec3> {
val t = Math.toRadians(orientation.toDouble())
val c = Math.cos(t).toFloat()
val s = Math.sin(t).toFloat()
var u = if (front) Vec3(-c, -s, 0f) else Vec3(c, -s, 0f)
var v = if (front) Vec3(s, -c, 0f) else Vec3(-s, -c, 0f)
repeat(Math.floorMod(turns, 4)) {
val nu = v * -1f
v = u
u = nu
}
return u to v
}
fun upInCamera(front: Boolean, orientation: Int, turns: Int, up: Vec3): Vec3 {
val (u, v) = axes(front, orientation, turns)
val n = up.normalized()
return Vec3(dot(n, u), dot(n, v), dot(n, u.cross(v)))
}
fun uprightTurns(front: Boolean, orientation: Int, up: Vec3): Int =
(0..3).minBy { turns -> dot(up.normalized(), axes(front, orientation, turns).second) }
fun placement(front: Boolean, up: Vec3): Placement {
val n = up.normalized()
if (n.length() < 0.5f) return Placement.UNKNOWN
if (n.z > FLAT) return Placement.FACE_UP
if (n.z < -FLAT) return Placement.FACE_DOWN
val elevation = Math.toDegrees(asin((if (front) n.z else -n.z).toDouble().coerceIn(-1.0, 1.0)))
return when {
elevation > STEEP -> Placement.POINTS_UP
elevation < -STEEP -> Placement.POINTS_DOWN
else -> Placement.STANDING
}
}
private fun dot(a: Vec3, b: Vec3) = a.x * b.x + a.y * b.y + a.z * b.z
private const val FLAT = 0.9f
private const val STEEP = 45.0
}
internal enum class Placement { UNKNOWN, STANDING, FACE_UP, FACE_DOWN, POINTS_UP, POINTS_DOWN }
@@ -47,6 +47,7 @@ object ControllerPrefs {
val qamButton: Boolean,
val keyboardButton: Boolean,
val selectSteam: Boolean,
val joyConVr: Boolean,
val snap: Boolean,
val customLayout: Boolean,
val mapping: Map<String, String>,
@@ -67,6 +68,7 @@ object ControllerPrefs {
qamButton = p.getBoolean("qamButton", true),
keyboardButton = p.getBoolean("keyboardButton", true),
selectSteam = p.getBoolean("selectSteam", false),
joyConVr = joyConVr(context),
snap = p.getBoolean("snap", true),
customLayout = p.all.keys.any { it.startsWith("layout.") },
mapping = mappable.associate { (id, _) -> id to target(context, id) },
@@ -127,6 +129,12 @@ object ControllerPrefs {
prefs(context).edit().putBoolean("selectSteam", on).apply()
}
fun joyConVr(context: Context): Boolean = prefs(context).getBoolean("joyConVr", true)
fun setJoyConVr(context: Context, on: Boolean) {
prefs(context).edit().putBoolean("joyConVr", on).apply()
}
fun setSnap(context: Context, on: Boolean) {
prefs(context).edit().putBoolean("snap", on).apply()
}
@@ -119,6 +119,10 @@ public class FakeInputWriter {
private static native void nativeStoreFence();
public static void storeFence() {
nativeStoreFence();
}
private final File eventFile;
private final int slot;
private final ByteBuffer buffer = ByteBuffer.allocateDirect(BUFFER_SIZE);
@@ -0,0 +1,143 @@
package com.droiddeck.launcher.input
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
object JoyCon {
const val LEFT = 0
const val RIGHT = 1
const val VR_TRIGGER = 1 shl 0
const val VR_GRIP = 1 shl 1
const val VR_A = 1 shl 2
const val VR_B = 1 shl 3
const val VR_STICK = 1 shl 4
const val VR_SYSTEM = 1 shl 5
const val VR_RUMBLE_SLOT = 0x100
private const val VENDOR = 0x057e
private const val PRODUCT_LEFT = 0x2006
private const val PRODUCT_RIGHT = 0x2007
private const val BTN_SOUTH = 0x130
private const val BTN_EAST = 0x131
private const val BTN_NORTH = 0x133
private const val BTN_WEST = 0x134
private const val BTN_Z = 0x135
private const val BTN_TL = 0x136
private const val BTN_TR = 0x137
private const val BTN_TL2 = 0x138
private const val BTN_TR2 = 0x139
private const val BTN_SELECT = 0x13a
private const val BTN_START = 0x13b
private const val BTN_MODE = 0x13c
private const val BTN_THUMBL = 0x13d
private const val BTN_THUMBR = 0x13e
private const val BTN_DPAD_UP = 0x220
private const val BTN_DPAD_DOWN = 0x221
private const val BTN_DPAD_LEFT = 0x222
private const val BTN_DPAD_RIGHT = 0x223
private val NO_VR_DEVICES = intArrayOf(PadBridge.NO_CONTROLLER, PadBridge.NO_CONTROLLER)
private var vrOwner: Any? = null
@Volatile private var vrDevices = NO_VR_DEVICES
@JvmStatic
fun side(device: InputDevice?): Int = when {
device == null || device.isVirtual || device.vendorId != VENDOR -> -1
device.productId == PRODUCT_LEFT -> LEFT
device.productId == PRODUCT_RIGHT -> RIGHT
else -> -1
}
@JvmStatic
fun stickAxes(side: Int, device: InputDevice, source: Int): IntArray = when {
side == LEFT -> intArrayOf(MotionEvent.AXIS_X, MotionEvent.AXIS_Y)
device.getMotionRange(MotionEvent.AXIS_RX, source) != null -> intArrayOf(MotionEvent.AXIS_RX, MotionEvent.AXIS_RY)
else -> intArrayOf(MotionEvent.AXIS_Z, MotionEvent.AXIS_RZ)
}
@JvmStatic
fun isRail(side: Int, scanCode: Int): Boolean = when (side) {
LEFT -> scanCode == BTN_TR || scanCode == BTN_TR2
RIGHT -> scanCode == BTN_TL || scanCode == BTN_TL2
else -> false
}
internal fun padKey(side: Int, scanCode: Int): Int = when (side) {
LEFT -> when (scanCode) {
BTN_TL -> KeyEvent.KEYCODE_BUTTON_L1
BTN_TL2 -> KeyEvent.KEYCODE_BUTTON_L2
BTN_SELECT -> KeyEvent.KEYCODE_BUTTON_SELECT
BTN_THUMBL -> KeyEvent.KEYCODE_BUTTON_THUMBL
BTN_Z -> KeyEvent.KEYCODE_BUTTON_Z
BTN_DPAD_UP -> KeyEvent.KEYCODE_DPAD_UP
BTN_DPAD_DOWN -> KeyEvent.KEYCODE_DPAD_DOWN
BTN_DPAD_LEFT -> KeyEvent.KEYCODE_DPAD_LEFT
BTN_DPAD_RIGHT -> KeyEvent.KEYCODE_DPAD_RIGHT
else -> KeyEvent.KEYCODE_UNKNOWN
}
RIGHT -> when (scanCode) {
BTN_SOUTH -> KeyEvent.KEYCODE_BUTTON_A
BTN_EAST -> KeyEvent.KEYCODE_BUTTON_B
BTN_NORTH -> KeyEvent.KEYCODE_BUTTON_Y
BTN_WEST -> KeyEvent.KEYCODE_BUTTON_X
BTN_TR -> KeyEvent.KEYCODE_BUTTON_R1
BTN_TR2 -> KeyEvent.KEYCODE_BUTTON_R2
BTN_START -> KeyEvent.KEYCODE_BUTTON_START
BTN_THUMBR -> KeyEvent.KEYCODE_BUTTON_THUMBR
BTN_MODE -> KeyEvent.KEYCODE_BUTTON_MODE
else -> KeyEvent.KEYCODE_UNKNOWN
}
else -> KeyEvent.KEYCODE_UNKNOWN
}
@JvmStatic
fun normalize(event: KeyEvent): KeyEvent {
val side = side(event.device)
if (side < 0) return event
val code = padKey(side, event.scanCode)
if (code == KeyEvent.KEYCODE_UNKNOWN || code == event.keyCode) return event
return KeyEvent(event.downTime, event.eventTime, event.action, code, event.repeatCount, event.metaState,
event.deviceId, event.scanCode, event.flags, event.source)
}
internal fun vrButton(side: Int, scanCode: Int): Int = when (side) {
LEFT -> when (scanCode) {
BTN_TL2 -> VR_TRIGGER
BTN_TL, BTN_TR, BTN_TR2 -> VR_GRIP
BTN_DPAD_DOWN, BTN_DPAD_RIGHT -> VR_A
BTN_DPAD_UP, BTN_DPAD_LEFT, BTN_SELECT -> VR_B
BTN_THUMBL -> VR_STICK
BTN_Z -> VR_SYSTEM
else -> 0
}
RIGHT -> when (scanCode) {
BTN_TR2 -> VR_TRIGGER
BTN_TR, BTN_TL, BTN_TL2 -> VR_GRIP
BTN_SOUTH, BTN_EAST -> VR_A
BTN_NORTH, BTN_WEST, BTN_START -> VR_B
BTN_THUMBR -> VR_STICK
BTN_MODE -> VR_SYSTEM
else -> 0
}
else -> 0
}
@JvmStatic
fun vrRumbleDevice(slot: Int): Int? =
if (slot == VR_RUMBLE_SLOT + LEFT || slot == VR_RUMBLE_SLOT + RIGHT) vrDevices[slot - VR_RUMBLE_SLOT] else null
@Synchronized
internal fun setVrDevices(owner: Any, devices: IntArray) {
vrOwner = owner
vrDevices = devices
}
@Synchronized
internal fun clearVrDevices(owner: Any) {
if (vrOwner !== owner) return
vrOwner = null
vrDevices = NO_VR_DEVICES
}
}
@@ -0,0 +1,213 @@
package com.droiddeck.launcher.input
import android.content.Context
import com.droiddeck.launcher.glasses.Quaternion
import com.droiddeck.launcher.glasses.Vec3
import kotlin.math.abs
import kotlin.math.exp
import kotlin.math.roundToInt
import kotlin.math.sqrt
internal class JoyConAccelCalibration {
var bias = Vec3(0f, 0f, 0f)
private set
var scale = 1f
private set
var ready = false
private set
private val sums = HashMap<Int, DoubleArray>()
fun restore(saved: FloatArray?) {
if (saved == null || saved.size != 4 || saved.any { it.isNaN() || it.isInfinite() }) return
val b = Vec3(saved[0], saved[1], saved[2])
if (!plausible(b, saved[3])) return
bias = b
scale = saved[3]
ready = true
}
fun saved(): FloatArray? = if (ready) floatArrayOf(bias.x, bias.y, bias.z, scale) else null
fun correct(raw: Vec3): Vec3 = if (ready) (raw - bias) * scale else raw
fun addResting(raw: Vec3): Boolean {
val n = raw.length()
if (n < 1e-3f) return false
val key = bin(raw.x / n) * 9 + bin(raw.y / n) * 3 + bin(raw.z / n)
val s = sums.getOrPut(key) { DoubleArray(4) }
s[0] += raw.x.toDouble()
s[1] += raw.y.toDouble()
s[2] += raw.z.toDouble()
s[3] += 1.0
return s[3] % SOLVE_EVERY == 0.0 && solve()
}
private fun solve(): Boolean {
val points = sums.values.filter { it[3] >= MIN_SAMPLES }.map { doubleArrayOf(it[0] / it[3], it[1] / it[3], it[2] / it[3]) }
if (points.size < MIN_BINS || smallestSpread(points) < MIN_SPREAD) return false
val ata = Array(4) { DoubleArray(4) }
val atb = DoubleArray(4)
for (p in points) {
val row = doubleArrayOf(2 * p[0], 2 * p[1], 2 * p[2], 1.0)
val rhs = p[0] * p[0] + p[1] * p[1] + p[2] * p[2]
for (i in 0..3) {
atb[i] += row[i] * rhs
for (j in 0..3) ata[i][j] += row[i] * row[j]
}
}
val x = solve4(ata, atb) ?: return false
val radius2 = x[3] + x[0] * x[0] + x[1] * x[1] + x[2] * x[2]
if (radius2 <= 0) return false
val b = Vec3(x[0].toFloat(), x[1].toFloat(), x[2].toFloat())
val s = (GRAVITY / sqrt(radius2)).toFloat()
if (!plausible(b, s)) return false
bias = b
scale = s
ready = true
return true
}
private fun bin(v: Float) = (v * 1.5f).roundToInt().coerceIn(-1, 1) + 1
companion object {
private const val GRAVITY = 9.80665
private const val MIN_SAMPLES = 20.0
private const val MIN_BINS = 5
private const val MIN_SPREAD = 2.0
private const val SOLVE_EVERY = 25.0
private const val MAX_BIAS = 1.5f
private const val MAX_SCALE_ERROR = 0.1f
private fun plausible(b: Vec3, s: Float) = b.length() <= MAX_BIAS && abs(s - 1f) <= MAX_SCALE_ERROR
private fun smallestSpread(points: List<DoubleArray>): Double {
val mean = DoubleArray(3) { i -> points.sumOf { it[i] } / points.size }
val c = Array(3) { DoubleArray(3) }
for (p in points) for (i in 0..2) for (j in 0..2) c[i][j] += (p[i] - mean[i]) * (p[j] - mean[j])
return sqrt(maxOf(smallestEigenvalue(c), 0.0))
}
private fun smallestEigenvalue(m: Array<DoubleArray>): Double {
val a = Array(3) { m[it].copyOf() }
repeat(50) {
var p = 0
var q = 1
for ((i, j) in listOf(0 to 2, 1 to 2)) if (abs(a[i][j]) > abs(a[p][q])) { p = i; q = j }
if (abs(a[p][q]) < 1e-12) return minOf(a[0][0], a[1][1], a[2][2])
val theta = (a[q][q] - a[p][p]) / (2 * a[p][q])
val t = (if (theta >= 0) 1.0 else -1.0) / (abs(theta) + sqrt(theta * theta + 1))
val c = 1 / sqrt(t * t + 1)
val s = t * c
for (k in 0..2) {
val kp = a[k][p]
val kq = a[k][q]
a[k][p] = c * kp - s * kq
a[k][q] = s * kp + c * kq
}
for (k in 0..2) {
val pk = a[p][k]
val qk = a[q][k]
a[p][k] = c * pk - s * qk
a[q][k] = s * pk + c * qk
}
}
return minOf(a[0][0], a[1][1], a[2][2])
}
private fun solve4(a: Array<DoubleArray>, b: DoubleArray): DoubleArray? {
val m = Array(4) { i -> DoubleArray(5) { j -> if (j < 4) a[i][j] else b[i] } }
for (col in 0..3) {
val pivot = (col..3).maxBy { abs(m[it][col]) }
if (abs(m[pivot][col]) < 1e-9) return null
val swap = m[col]
m[col] = m[pivot]
m[pivot] = swap
for (r in 0..3) {
if (r == col) continue
val f = m[r][col] / m[col][col]
for (k in col..4) m[r][k] -= f * m[col][k]
}
}
return DoubleArray(4) { m[it][4] / m[it][it] }
}
}
}
internal class JoyConMotion {
var velocity = Vec3(0f, 0f, 0f)
private set
var displacement = Vec3(0f, 0f, 0f)
private set
var inertial = Vec3(0f, 0f, 0f)
private set
var resting = false
private set
private var gravity = Float.NaN
private var average = Vec3(0f, 0f, 0f)
private var calmFor = 0f
private var stillFor = 0f
private var movingFor = 0f
fun update(rotation: Quaternion, accel: Vec3, rate: Vec3, calibrated: Boolean, dt: Float) {
if (dt <= 0f) return
if (gravity.isNaN()) {
gravity = accel.length()
average = accel
}
if (calibrated) gravity = GRAVITY
average += (accel - average) * (dt / AVERAGE_SECONDS).coerceAtMost(1f)
val world = rotation.rotate(accel)
val linear = Vec3(world.x, world.y - gravity, world.z)
inertial = inertial * exp(-dt / INERTIAL_SECONDS) + linear * dt
val calm = rate.length() < CALM_RATE && (accel - average).length() < CALM_ACCEL
calmFor = if (calm) calmFor + dt else 0f
val still = rate.length() < CALM_RATE &&
(linear.length() < STILL_ACCEL || calmFor >= if (calibrated) CALIBRATED_CALM_SECONDS else UNCALIBRATED_CALM_SECONDS)
stillFor = if (still) stillFor + dt else 0f
resting = calmFor >= RESTING_SECONDS && rate.length() < RESTING_RATE
if (stillFor >= STILL_SECONDS) {
movingFor = 0f
velocity *= exp(-dt / SETTLE_SECONDS)
if (!calibrated) gravity += (accel.length() - gravity) * (dt / GRAVITY_SECONDS).coerceAtMost(1f)
} else {
movingFor += dt
val leak = if (movingFor < FREE_SECONDS) LEAK_SECONDS else LONG_MOVE_LEAK_SECONDS
velocity = velocity * exp(-dt / leak) + linear * dt
}
displacement += velocity * dt
}
companion object {
private const val GRAVITY = 9.80665f
private const val AVERAGE_SECONDS = 0.1f
private const val CALM_RATE = 0.2f
private const val CALM_ACCEL = 0.25f
private const val STILL_ACCEL = 0.12f
private const val STILL_SECONDS = 0.2f
private const val UNCALIBRATED_CALM_SECONDS = 0.5f
private const val CALIBRATED_CALM_SECONDS = 2f
private const val SETTLE_SECONDS = 0.05f
private const val LEAK_SECONDS = 20f
private const val FREE_SECONDS = 1f
private const val LONG_MOVE_LEAK_SECONDS = 0.2f
private const val GRAVITY_SECONDS = 0.5f
private const val INERTIAL_SECONDS = 30f
private const val RESTING_SECONDS = 0.3f
private const val RESTING_RATE = 0.05f
}
}
internal object JoyConCalibrations {
fun load(context: Context, descriptor: String?): FloatArray? {
if (descriptor.isNullOrEmpty()) return null
val text = ControllerPrefs.prefs(context).getString(KEY + descriptor, null) ?: return null
return text.split(',').mapNotNull { it.toFloatOrNull() }.toFloatArray().takeIf { it.size == 4 }
}
fun save(context: Context, descriptor: String?, values: FloatArray?) {
if (descriptor.isNullOrEmpty() || values == null) return
ControllerPrefs.prefs(context).edit().putString(KEY + descriptor, values.joinToString(",")).apply()
}
private const val KEY = "joyConAccel."
}
@@ -0,0 +1,113 @@
package com.droiddeck.launcher.input
import com.droiddeck.launcher.glasses.Quaternion
import com.droiddeck.launcher.glasses.Vec3
import kotlin.math.abs
internal class JoyConOrientation {
var rotation = Quaternion.IDENTITY
private set
var rate = Vec3(0f, 0f, 0f)
private set
var ready = false
private set
private var bias = Vec3(0f, 0f, 0f)
private var stillFor = 0f
fun update(accelKernel: FloatArray, gyroKernel: FloatArray, dt: Float) = update(axes(accelKernel), axes(gyroKernel), dt)
fun update(accel: Vec3, gyro: Vec3, dt: Float) {
val norm = accel.length()
val gravityValid = abs(norm - GRAVITY) < GRAVITY_TOLERANCE
if (!ready) {
if (!gravityValid) return
val up = accel.normalized()
val axis = up.cross(WORLD_UP)
rotation = if (up.y < -0.9999f) Quaternion.fromAxisAngle(Vec3(1f, 0f, 0f), Math.PI.toFloat())
else Quaternion(1f + up.y, axis.x, axis.y, axis.z).normalized()
rate = Vec3(0f, 0f, 0f)
ready = true
return
}
if (dt <= 0f) return
val still = gravityValid && abs(norm - GRAVITY) < STILL_ACCEL && (gyro - bias).length() < STILL_RATE
stillFor = if (still) stillFor + dt else 0f
if (stillFor >= STILL_SECONDS) {
val alpha = (dt / BIAS_SECONDS).coerceAtMost(1f)
bias = bias * (1f - alpha) + gyro * alpha
}
rate = gyro - bias
var corrected = rate
if (gravityValid && rate.length() < MOVING_RATE) {
val predictedUp = rotation.conjugate().rotate(WORLD_UP)
corrected += accel.normalized().cross(predictedUp) * (GAIN * (1f - abs(norm - GRAVITY) / GRAVITY_TOLERANCE))
}
rotation = rotation.integrateBodyRate(corrected, dt).normalized()
}
companion object {
private val WORLD_UP = Vec3(0f, 1f, 0f)
private const val GRAVITY = 9.80665f
private const val GRAVITY_TOLERANCE = 1.0f
private const val STILL_ACCEL = 0.5f
private const val STILL_RATE = 0.05f
private const val STILL_SECONDS = 0.3f
private const val BIAS_SECONDS = 2f
private const val GAIN = 1.0f
private const val MOVING_RATE = 1.0f
fun axes(v: FloatArray) = Vec3(-v[1], v[2], -v[0])
}
}
internal class JoyConReports {
class Sample(val accel: FloatArray, val gyro: FloatArray, val dt: Float)
private val pending = ArrayList<FloatArray>(4)
private var pendingAt = 0L
private var previousFirst: FloatArray? = null
private var previousLast: FloatArray? = null
private var previousSample: FloatArray? = null
private var previousAt = 0L
private var order = NEWEST_FIRST_START
val size: Int get() = pending.size
fun add(accel: FloatArray, gyro: FloatArray, arrivalNs: Long) {
if (pending.isEmpty()) pendingAt = arrivalNs
pending += floatArrayOf(accel[0], accel[1], accel[2], gyro[0], gyro[1], gyro[2])
}
fun flush(): List<Sample> {
if (pending.isEmpty()) return emptyList()
val first = pending.first()
val last = pending.last()
if (pending.size > 1) {
if (previousFirst?.contentEquals(last) == true) order = (order + 1).coerceAtMost(ORDER_LIMIT)
else if (previousLast?.contentEquals(first) == true) order = (order - 1).coerceAtLeast(-ORDER_LIMIT)
}
val chronological = if (order >= 0) pending.asReversed() else pending
val fresh = ArrayList<FloatArray>(chronological.size)
var before = previousSample
for (sample in chronological) {
if (before?.contentEquals(sample) != true) fresh += sample
before = sample
}
val used = fresh.ifEmpty { listOf(chronological.last()) }
val dt = if (previousAt == 0L) 0f else ((pendingAt - previousAt).coerceAtLeast(0L) * 1e-9f / used.size).coerceAtMost(MAX_STEP_S)
val out = used.map { Sample(it.copyOfRange(0, 3), it.copyOfRange(3, 6), dt) }
previousFirst = first
previousLast = last
previousSample = chronological.last()
previousAt = pendingAt
pending.clear()
return out
}
companion object {
const val SAMPLES_PER_REPORT = 3
private const val NEWEST_FIRST_START = 2
private const val ORDER_LIMIT = 20
private const val MAX_STEP_S = 0.05f
}
}
@@ -0,0 +1,116 @@
package com.droiddeck.launcher.input
import android.util.Log
import com.droiddeck.launcher.glasses.Quaternion
import com.droiddeck.launcher.glasses.Vec3
import java.io.File
import java.io.RandomAccessFile
import java.nio.ByteOrder
import java.nio.MappedByteBuffer
import java.nio.channels.FileChannel
internal class JoyConShm private constructor(
private val file: RandomAccessFile,
private val map: MappedByteBuffer,
private val fence: () -> Unit,
) {
class Hand(
val connected: Boolean,
val charging: Boolean = false,
val buttons: Int = 0,
val rotation: Quaternion = Quaternion.IDENTITY,
val rate: Vec3 = Vec3(0f, 0f, 0f),
val displacement: Vec3 = Vec3(0f, 0f, 0f),
val velocity: Vec3 = Vec3(0f, 0f, 0f),
val inertial: Vec3 = Vec3(0f, 0f, 0f),
val stickX: Float = 0f,
val stickY: Float = 0f,
val battery: Float = -1f,
val recenter: Int = 0,
val timeNs: Long = 0L,
)
private var seq = map.getInt(SEQ).let { if (it and 1 != 0) it + 1 else it }
init {
begin()
map.putInt(MAGIC_AT, MAGIC)
map.putInt(VERSION_AT, VERSION)
for (at in HANDS until SIZE step 4) map.putInt(at, 0)
end()
}
fun write(side: Int, hand: Hand) {
val at = HANDS + side * HAND_SIZE
begin()
map.putInt(at, (if (hand.connected) CONNECTED else 0) or (if (hand.charging) CHARGING else 0))
map.putInt(at + 4, hand.buttons)
map.putFloat(at + 8, hand.rotation.w)
map.putFloat(at + 12, hand.rotation.x)
map.putFloat(at + 16, hand.rotation.y)
map.putFloat(at + 20, hand.rotation.z)
map.putFloat(at + 24, hand.rate.x)
map.putFloat(at + 28, hand.rate.y)
map.putFloat(at + 32, hand.rate.z)
map.putFloat(at + 36, hand.stickX)
map.putFloat(at + 40, hand.stickY)
map.putFloat(at + 44, hand.battery)
map.putInt(at + 48, hand.recenter)
map.putLong(at + 56, hand.timeNs)
map.putFloat(at + 64, hand.displacement.x)
map.putFloat(at + 68, hand.displacement.y)
map.putFloat(at + 72, hand.displacement.z)
map.putFloat(at + 76, hand.velocity.x)
map.putFloat(at + 80, hand.velocity.y)
map.putFloat(at + 84, hand.velocity.z)
map.putFloat(at + 88, hand.inertial.x)
map.putFloat(at + 92, hand.inertial.y)
map.putFloat(at + 96, hand.inertial.z)
map.putInt(at + 100, 0)
end()
}
fun close() {
write(JoyCon.LEFT, Hand(connected = false))
write(JoyCon.RIGHT, Hand(connected = false))
runCatching { file.close() }
}
private fun begin() {
map.putInt(SEQ, ++seq)
fence()
}
private fun end() {
fence()
map.putInt(SEQ, ++seq)
}
companion object {
const val FILE = "droiddeck-joycon"
const val SIZE = 224
private const val MAGIC = 0x4E434A44
private const val VERSION = 3
private const val MAGIC_AT = 0
private const val VERSION_AT = 4
private const val SEQ = 8
private const val HANDS = 16
private const val HAND_SIZE = 104
private const val CONNECTED = 1
private const val CHARGING = 2
fun open(path: File, fence: () -> Unit = FakeInputWriter::storeFence): JoyConShm? = runCatching {
path.parentFile?.mkdirs()
val file = RandomAccessFile(path, "rw")
try {
file.setLength(SIZE.toLong())
val map = file.channel.map(FileChannel.MapMode.READ_WRITE, 0, SIZE.toLong())
map.order(ByteOrder.LITTLE_ENDIAN)
JoyConShm(file, map, fence)
} catch (e: Exception) {
file.close()
throw e
}
}.onFailure { Log.w("JoyConVr", "Joy-Con pose file unavailable: $path", it) }.getOrNull()
}
}
@@ -0,0 +1,363 @@
package com.droiddeck.launcher.input
import android.content.Context
import android.hardware.BatteryState
import android.hardware.Sensor
import android.hardware.SensorEvent
import android.hardware.SensorEventListener
import android.hardware.SensorManager
import android.os.CombinedVibration
import android.os.Handler
import android.os.HandlerThread
import android.os.Looper
import android.os.Process
import android.os.VibrationEffect
import android.util.Log
import android.view.InputDevice
import android.view.KeyEvent
import android.view.MotionEvent
import androidx.annotation.RequiresApi
import java.io.File
import kotlin.math.min
import kotlin.math.sqrt
@RequiresApi(31)
class JoyConVr(
private val context: Context,
private val path: File,
private val onDevicesChanged: (taken: List<Int>, released: List<Int>) -> Unit,
) {
private val thread = HandlerThread("joycon-vr", Process.THREAD_PRIORITY_DISPLAY).apply { start() }
private val worker = Handler(thread.looper)
private val main = Handler(Looper.getMainLooper())
@Volatile private var owned = IntArray(0)
private var wanted = false
private var closed = false
private var retries = 0
private var shm: JoyConShm? = null
private val hands = arrayOfNulls<Hand>(2)
private val retry = Runnable { apply() }
fun setActive(on: Boolean) {
worker.post {
if (closed || wanted == on) return@post
wanted = on
retries = 0
apply()
}
}
fun refresh() {
worker.post {
if (closed) return@post
retries = 0
apply()
}
}
fun owns(deviceId: Int): Boolean = owned.contains(deviceId)
val holding: Boolean get() = owned.isNotEmpty()
fun onKeyEvent(event: KeyEvent): Boolean {
if (!owns(event.deviceId)) return false
if (event.repeatCount > 0) return true
val down = when (event.action) {
KeyEvent.ACTION_DOWN -> true
KeyEvent.ACTION_UP -> false
else -> return true
}
val id = event.deviceId
val scanCode = event.scanCode
worker.post { hand(id)?.key(scanCode, down) }
return true
}
fun onMotionEvent(event: MotionEvent): Boolean {
if (!owns(event.deviceId)) return false
if (!event.isFromSource(InputDevice.SOURCE_JOYSTICK) || event.actionMasked != MotionEvent.ACTION_MOVE) return true
val device = event.device ?: return true
val side = JoyCon.side(device)
if (side < 0) return true
val axes = JoyCon.stickAxes(side, device, event.source)
val x = event.getAxisValue(axes[0])
val y = event.getAxisValue(axes[1])
val id = event.deviceId
worker.post { hand(id)?.stick(x, -y) }
return true
}
fun releaseButtons() {
worker.post { hands.forEach { it?.release() } }
}
fun close() {
worker.post {
closed = true
wanted = false
apply()
shm?.close()
shm = null
}
thread.quitSafely()
}
private fun hand(id: Int): Hand? = hands.firstOrNull { it?.device?.id == id }
private fun apply() {
worker.removeCallbacks(retry)
val found = arrayOfNulls<InputDevice>(2)
var waiting = false
if (wanted && !closed) {
for (id in InputDevice.getDeviceIds()) {
val device = InputDevice.getDevice(id) ?: continue
val side = JoyCon.side(device)
if (side < 0 || found[side] != null) continue
if (hasMotion(device)) found[side] = device else waiting = true
}
}
val before = owned
for (side in 0..1) {
val current = hands[side]
val device = found[side]
if (current != null && current.device.id == device?.id) continue
current?.stop()
hands[side] = null
if (device != null) {
hands[side] = start(side, device)
if (hands[side] == null) waiting = true
}
}
val now = hands.mapNotNull { it?.device?.id }.toIntArray()
owned = now
if (closed) JoyCon.clearVrDevices(this)
else JoyCon.setVrDevices(this, IntArray(2) { hands[it]?.device?.id ?: PadBridge.NO_CONTROLLER })
if (!closed) {
val taken = now.filter { it !in before }
val released = before.filter { it !in now }
if (taken.isNotEmpty() || released.isNotEmpty()) main.post { onDevicesChanged(taken, released) }
}
if (wanted && waiting && retries < MAX_RETRIES) {
retries++
worker.postDelayed(retry, RETRY_MS)
}
}
private fun hasMotion(device: InputDevice): Boolean =
runCatching {
val sensors = device.sensorManager
sensors.getDefaultSensor(Sensor.TYPE_GYROSCOPE) != null && sensors.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null
}.getOrDefault(false)
private fun start(side: Int, device: InputDevice): Hand? {
val out = shm ?: JoyConShm.open(path)?.also { shm = it } ?: return null
val sensors = runCatching { device.sensorManager }.getOrNull() ?: return null
val gyro = runCatching { sensors.getDefaultSensor(Sensor.TYPE_GYROSCOPE) }.getOrNull() ?: return null
val accel = runCatching { sensors.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) }.getOrNull() ?: return null
val hand = Hand(side, device, sensors, out)
val listening = runCatching {
sensors.registerListener(hand, accel, SensorManager.SENSOR_DELAY_FASTEST, worker) &&
sensors.registerListener(hand, gyro, SensorManager.SENSOR_DELAY_FASTEST, worker)
}.onFailure { Log.w(TAG, "Joy-Con motion unavailable for \"${device.name}\"", it) }.getOrDefault(false)
if (!listening) {
runCatching { sensors.unregisterListener(hand) }
return null
}
Log.i(TAG, "${SIDES[side]} Joy-Con \"${device.name}\" (id ${device.id}) is a VR controller")
return hand
}
private inner class Hand(
val side: Int,
val device: InputDevice,
private val sensors: SensorManager,
private val out: JoyConShm,
) : SensorEventListener {
private val filter = JoyConOrientation()
private val motion = JoyConMotion()
private val calibration = JoyConAccelCalibration().apply { restore(JoyConCalibrations.load(context, device.descriptor)) }
private val reports = JoyConReports()
private val accel = FloatArray(3)
private var accelStamp = 0L
private var haveAccel = false
private val pressed = HashSet<Int>()
private var stickX = 0f
private var stickY = 0f
private var systemDown = false
private var systemTap = false
private var held = false
private var recenter = 0
private var battery = -1f
private var charging = false
private var batteryAt = 0L
private var sampleAt = 0L
private var live = true
private val hold = Runnable {
if (!systemDown || held) return@Runnable
held = true
recenter++
confirm()
publish()
}
private val tapEnd = Runnable {
systemTap = false
publish()
}
private val flush = Runnable { flushReport() }
override fun onSensorChanged(event: SensorEvent) {
if (!live) return
when (event.sensor.type) {
Sensor.TYPE_ACCELEROMETER -> {
if (event.timestamp == accelStamp) return
accelStamp = event.timestamp
event.values.copyInto(accel, 0, 0, 3)
haveAccel = true
}
Sensor.TYPE_GYROSCOPE -> {
if (!haveAccel) return
reports.add(accel, event.values, System.nanoTime())
worker.removeCallbacks(flush)
if (reports.size >= JoyConReports.SAMPLES_PER_REPORT) flushReport() else worker.postDelayed(flush, FLUSH_MS)
}
}
}
private fun flushReport() {
val samples = reports.flush()
if (samples.isEmpty() || !live) return
for (sample in samples) {
val raw = JoyConOrientation.axes(sample.accel)
val corrected = calibration.correct(raw)
filter.update(corrected, JoyConOrientation.axes(sample.gyro), sample.dt)
if (!filter.ready) continue
motion.update(filter.rotation, corrected, filter.rate, calibration.ready, sample.dt)
if (motion.resting && calibration.addResting(raw)) JoyConCalibrations.save(context, device.descriptor, calibration.saved())
}
val now = System.nanoTime()
sampleAt = now
if (now - batteryAt > BATTERY_NS) refreshBattery(now)
publish()
}
override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) = Unit
fun key(scanCode: Int, down: Boolean) {
if (!live) return
val bit = JoyCon.vrButton(side, scanCode)
if (bit == 0) return
if (bit == JoyCon.VR_SYSTEM) {
system(down)
return
}
if (down) pressed += scanCode else pressed -= scanCode
publish()
}
fun stick(x: Float, y: Float) {
if (!live) return
val length = sqrt(x * x + y * y)
if (length < STICK_DEADZONE) {
stickX = 0f
stickY = 0f
} else {
val scale = min((length - STICK_DEADZONE) / (1f - STICK_DEADZONE), 1f) / length
stickX = x * scale
stickY = y * scale
}
publish()
}
fun release() {
if (!live) return
pressed.clear()
stickX = 0f
stickY = 0f
systemDown = false
systemTap = false
held = false
worker.removeCallbacks(hold)
worker.removeCallbacks(tapEnd)
publish()
}
fun stop() {
live = false
runCatching { sensors.unregisterListener(this) }
worker.removeCallbacks(hold)
worker.removeCallbacks(tapEnd)
worker.removeCallbacks(flush)
out.write(side, JoyConShm.Hand(connected = false))
Log.i(TAG, "${SIDES[side]} Joy-Con \"${device.name}\" (id ${device.id}) is a game pad again")
}
private fun system(down: Boolean) {
if (down == systemDown) return
systemDown = down
if (down) {
held = false
worker.postDelayed(hold, HOLD_MS)
return
}
worker.removeCallbacks(hold)
if (held) return
systemTap = true
publish()
worker.removeCallbacks(tapEnd)
worker.postDelayed(tapEnd, TAP_MS)
}
private fun buttons(): Int {
var bits = if (systemTap) JoyCon.VR_SYSTEM else 0
for (scanCode in pressed) bits = bits or JoyCon.vrButton(side, scanCode)
return bits
}
private fun publish() {
out.write(side, JoyConShm.Hand(
connected = filter.ready,
charging = charging,
buttons = buttons(),
rotation = filter.rotation,
rate = filter.rate,
displacement = motion.displacement,
velocity = motion.velocity,
inertial = motion.inertial,
stickX = stickX,
stickY = stickY,
battery = battery,
recenter = recenter,
timeNs = sampleAt,
))
}
private fun refreshBattery(now: Long) {
batteryAt = now
val state = runCatching { device.batteryState }.getOrNull() ?: return
battery = if (state.isPresent && !state.capacity.isNaN()) state.capacity else -1f
charging = state.isPresent && state.status == BatteryState.STATUS_CHARGING
}
private fun confirm() {
if (!ControllerPrefs.rumbleEnabled(context)) return
runCatching {
device.vibratorManager.vibrate(CombinedVibration.createParallel(
VibrationEffect.createOneShot(CONFIRM_MS, VibrationEffect.DEFAULT_AMPLITUDE)))
}
}
}
companion object {
const val FILE = JoyConShm.FILE
private const val TAG = "JoyConVr"
private val SIDES = arrayOf("left", "right")
private const val MAX_RETRIES = 6
private const val RETRY_MS = 500L
private const val HOLD_MS = 800L
private const val TAP_MS = 120L
private const val CONFIRM_MS = 80L
private const val BATTERY_NS = 30_000_000_000L
private const val STICK_DEADZONE = 0.12f
private const val FLUSH_MS = 3L
}
}
@@ -92,6 +92,7 @@ class OnScreenControls(
private val groups = controls.map { it.group }.distinct()
private var buttonsOnly = false
val isButtonsOnly: Boolean get() = buttonsOnly
private var quickHidden = false
private var quickPressedBy = -1
private var keyboardPressedBy = -1
@@ -30,6 +30,7 @@ public final class PadBridge {
KeyEvent.KEYCODE_BUTTON_THUMBL, KeyEvent.KEYCODE_BUTTON_THUMBR,
KeyEvent.KEYCODE_BUTTON_MODE, KeyEvent.KEYCODE_BUTTON_L2, KeyEvent.KEYCODE_BUTTON_R2
};
private static final int[] ROUTED_PASS_THROUGH = {PadState.GUIDE, PadState.QAM, PadState.SELECT};
private static final long QAM_GUIDE_LEAD_MIN_MS = 80;
private static final long QAM_GUIDE_LEAD_MAX_MS = 1500;
private static final int QAM_GUIDE_LEAD_FRAMES = 8;
@@ -46,12 +47,16 @@ public final class PadBridge {
private final FakeInputWriter writer;
private final PadState state = new PadState();
private final int[] owners = new int[SLOTS];
private final int[] partners = new int[SLOTS];
private volatile java.util.function.IntPredicate excluded = id -> false;
private final FakeInputWriter[] playerWriters = new FakeInputWriter[SLOTS];
private final PadState[] playerStates = new PadState[SLOTS];
private final PadState effectiveState = new PadState();
private final Handler mainHandler = new Handler(Looper.getMainLooper());
private boolean open;
private volatile boolean selectSteam;
private TouchRoute touchRoute;
private final PadState routedTouch = new PadState();
private boolean systemGuidePressed;
private boolean systemQamPressed;
private boolean qamChordActive;
@@ -72,8 +77,10 @@ public final class PadBridge {
public static final int NO_CONTROLLER = -1;
private static volatile int activeControllerId = NO_CONTROLLER;
private static final AtomicIntegerArray slotControllers = new AtomicIntegerArray(SLOTS);
private static final AtomicIntegerArray slotJoyCons = new AtomicIntegerArray(SLOTS * 2);
static {
for (int slot = 0; slot < SLOTS; slot++) slotControllers.set(slot, NO_CONTROLLER);
for (int i = 0; i < SLOTS * 2; i++) slotJoyCons.set(i, NO_CONTROLLER);
}
public static int activeControllerId() { return activeControllerId; }
@@ -82,10 +89,16 @@ public final class PadBridge {
return slot >= 0 && slot < SLOTS ? slotControllers.get(slot) : activeControllerId;
}
public static int joyConForSlot(int slot, int side) {
return slot >= 0 && slot < SLOTS && (side == JoyCon.LEFT || side == JoyCon.RIGHT)
? slotJoyCons.get(slot * 2 + side) : NO_CONTROLLER;
}
public PadBridge(File fakeInputDir) {
this.fakeInputDir = fakeInputDir;
writer = new FakeInputWriter(fakeInputDir.getAbsolutePath(), 0);
Arrays.fill(owners, NO_CONTROLLER);
Arrays.fill(partners, NO_CONTROLLER);
for (int slot = 1; slot < SLOTS; slot++) playerStates[slot] = new PadState();
}
@@ -93,6 +106,10 @@ public final class PadBridge {
selectSteam = on;
}
public void setExcluded(java.util.function.IntPredicate devices) {
excluded = devices;
}
public synchronized boolean start() {
if (!open) {
open = writer.open();
@@ -106,7 +123,9 @@ public final class PadBridge {
activeControllerId = NO_CONTROLLER;
for (int slot = 0; slot < SLOTS; slot++) {
owners[slot] = NO_CONTROLLER;
partners[slot] = NO_CONTROLLER;
slotControllers.set(slot, NO_CONTROLLER);
clearJoyCons(slot);
if (playerWriters[slot] != null) {
playerWriters[slot].destroy();
playerWriters[slot] = null;
@@ -206,7 +225,7 @@ public final class PadBridge {
public synchronized void syncConnectedControllers() {
for (int deviceId : InputDevice.getDeviceIds()) {
InputDevice device = InputDevice.getDevice(deviceId);
if (isFromController(device)) publish(slotFor(device));
if (isFromController(device) && !excluded.test(deviceId)) publish(slotFor(device));
}
}
@@ -224,8 +243,10 @@ public final class PadBridge {
}
public synchronized boolean onKeyEvent(KeyEvent event) {
if (!isControllerKeyEvent(event)) return false;
if (!isControllerKeyEvent(event) || excluded.test(event.getDeviceId())) return false;
int slot = slotFor(event.getDevice());
int side = JoyCon.side(event.getDevice());
if (side >= 0 && partners[slot] != NO_CONTROLLER && JoyCon.isRail(side, event.getScanCode())) return true;
PadState pad = stateFor(slot);
boolean pressed = event.getAction() == KeyEvent.ACTION_DOWN;
switch (event.getKeyCode()) {
@@ -269,36 +290,37 @@ public final class PadBridge {
}
public synchronized boolean onMotionEvent(MotionEvent event) {
if (!isControllerMotionEvent(event)) return false;
if (!isControllerMotionEvent(event) || excluded.test(event.getDeviceId())) return false;
if (event.getAction() != MotionEvent.ACTION_MOVE) return false;
int slot = slotFor(event.getDevice());
PadState pad = stateFor(slot);
noteDevice(event.getDevice(), slot);
statAxes++;
scheduleStats();
InputDevice device = event.getDevice();
int side = JoyCon.side(device);
if (side >= 0) {
int[] stick = JoyCon.stickAxes(side, device, event.getSource());
float x = axis(event, stick[0]);
float y = axis(event, stick[1]);
if (side == JoyCon.LEFT) {
pad.leftX = x;
pad.leftY = y;
} else {
pad.rightX = x;
pad.rightY = y;
}
if (Math.max(Math.abs(x), Math.abs(y)) > 0.5f) notePlayerInput();
publish(slot);
return true;
}
pad.leftX = axis(event, MotionEvent.AXIS_X);
pad.leftY = axis(event, MotionEvent.AXIS_Y);
InputDevice device = event.getDevice();
boolean wiredXboxLayout = device != null
&& ((device.getVendorId() == 0x20d6 && device.getProductId() == 0x2063)
|| (device.getVendorId() == 0x0e6f && device.getProductId() == 0x02f0))
&& device.getMotionRange(MotionEvent.AXIS_RX, event.getSource()) != null
&& device.getMotionRange(MotionEvent.AXIS_RY, event.getSource()) != null
&& device.getMotionRange(MotionEvent.AXIS_Z, event.getSource()) != null
&& device.getMotionRange(MotionEvent.AXIS_RZ, event.getSource()) != null;
pad.rightX = axis(event, wiredXboxLayout ? MotionEvent.AXIS_RX : MotionEvent.AXIS_Z);
pad.rightY = axis(event, wiredXboxLayout ? MotionEvent.AXIS_RY : MotionEvent.AXIS_RZ);
float lt;
float rt;
if (wiredXboxLayout) {
lt = Math.max(0f, Math.min(1f, (event.getAxisValue(MotionEvent.AXIS_Z) + 1f) / 2f));
rt = Math.max(0f, Math.min(1f, (event.getAxisValue(MotionEvent.AXIS_RZ) + 1f) / 2f));
} else {
lt = event.getAxisValue(MotionEvent.AXIS_LTRIGGER);
rt = event.getAxisValue(MotionEvent.AXIS_RTRIGGER);
if (lt == 0f) lt = event.getAxisValue(MotionEvent.AXIS_BRAKE);
if (rt == 0f) rt = event.getAxisValue(MotionEvent.AXIS_GAS);
}
boolean wired = wiredXboxLayout(event);
pad.rightX = rightX(event, wired);
pad.rightY = rightY(event, wired);
float lt = leftTrigger(event, wired);
float rt = rightTrigger(event, wired);
pad.leftTrigger = lt;
pad.rightTrigger = rt;
float hatX = event.getAxisValue(MotionEvent.AXIS_HAT_X);
@@ -315,7 +337,36 @@ public final class PadBridge {
return true;
}
public interface TouchRoute {
boolean active();
void accept(PadState state);
}
public synchronized void setTouchRoute(TouchRoute route) {
touchRoute = route;
routedTouch.clear();
}
public synchronized void applyTouch(java.util.function.Consumer<PadState> mutation) {
if (touchRoute != null && touchRoute.active()) {
mutation.accept(routedTouch);
touchRoute.accept(routedTouch);
boolean changed = false;
for (int button : ROUTED_PASS_THROUGH) {
boolean down = routedTouch.isDown(button);
if (state.isDown(button) == down) continue;
state.press(button, down);
changed = true;
}
if (changed) {
activeControllerId = NO_CONTROLLER;
slotControllers.set(0, NO_CONTROLLER);
publish();
}
return;
}
routedTouch.clear();
mutation.accept(state);
activeControllerId = NO_CONTROLLER;
slotControllers.set(0, NO_CONTROLLER);
@@ -346,6 +397,7 @@ public final class PadBridge {
}
public synchronized void releaseAll() {
routedTouch.clear();
releaseBackPaddles();
state.clear();
publish();
@@ -358,7 +410,7 @@ public final class PadBridge {
public synchronized void onDeviceRemoved(int deviceId) {
for (int slot = 0; slot < SLOTS; slot++) {
if (owners[slot] == deviceId) free(slot);
if (owners[slot] == deviceId || partners[slot] == deviceId) leave(slot, deviceId);
}
}
@@ -425,12 +477,26 @@ public final class PadBridge {
private int slotFor(InputDevice device) {
int id = device.getId();
if (owners[0] == id) return 0;
if (owners[0] == id || partners[0] == id) return 0;
for (int slot = 1; slot < SLOTS; slot++) {
if (owners[slot] == id) return slot;
if (owners[slot] == id || partners[slot] == id) return slot;
}
for (int slot = 0; slot < SLOTS; slot++) {
if (owners[slot] != NO_CONTROLLER && InputDevice.getDevice(owners[slot]) == null) free(slot);
if (partners[slot] != NO_CONTROLLER && InputDevice.getDevice(partners[slot]) == null) leave(slot, partners[slot]);
if (owners[slot] != NO_CONTROLLER && InputDevice.getDevice(owners[slot]) == null) leave(slot, owners[slot]);
}
int side = JoyCon.side(device);
for (int slot = 0; side >= 0 && slot < SLOTS; slot++) {
if (owners[slot] == NO_CONTROLLER || partners[slot] != NO_CONTROLLER) continue;
InputDevice owner = InputDevice.getDevice(owners[slot]);
if (JoyCon.side(owner) != 1 - side) continue;
partners[slot] = id;
slotJoyCons.set(slot * 2 + side, id);
slotJoyCons.set(slot * 2 + 1 - side, owner.getId());
if (slot == 0) releaseBackPaddles();
stateFor(slot).clear();
Log.i(TAG, "\"" + device.getName() + "\" (id " + id + ") joins \"" + owner.getName() + "\" as player " + (slot + 1));
return slot;
}
for (int slot = 0; slot < SLOTS; slot++) {
if (owners[slot] != NO_CONTROLLER) continue;
@@ -446,7 +512,8 @@ public final class PadBridge {
}
private boolean isPrimary(KeyEvent event) {
return owners[0] == event.getDevice().getId();
int id = event.getDevice().getId();
return owners[0] == id || partners[0] == id;
}
private void releaseBackPaddles() {
@@ -454,9 +521,32 @@ public final class PadBridge {
DeckControls.INSTANCE.setGrip(DeckControls.R4, false);
}
private void leave(int slot, int id) {
if (partners[slot] == NO_CONTROLLER) {
free(slot);
return;
}
Log.i(TAG, "player " + (slot + 1) + " Joy-Con (id " + id + ") left");
if (owners[slot] == id) owners[slot] = partners[slot];
partners[slot] = NO_CONTROLLER;
clearJoyCons(slot);
if (slotControllers.get(slot) == id) slotControllers.set(slot, owners[slot]);
if (activeControllerId == id) activeControllerId = owners[slot];
if (slot == 0) releaseBackPaddles();
stateFor(slot).clear();
publish(slot);
}
private static void clearJoyCons(int slot) {
slotJoyCons.set(slot * 2, NO_CONTROLLER);
slotJoyCons.set(slot * 2 + 1, NO_CONTROLLER);
}
private void free(int slot) {
Log.i(TAG, "player " + (slot + 1) + " (id " + owners[slot] + ") left");
owners[slot] = NO_CONTROLLER;
partners[slot] = NO_CONTROLLER;
clearJoyCons(slot);
slotControllers.set(slot, NO_CONTROLLER);
if (slot == 0) {
releaseBackPaddles();
@@ -535,6 +625,37 @@ public final class PadBridge {
writer.writePad(effectiveState);
}
public static boolean wiredXboxLayout(MotionEvent event) {
InputDevice device = event.getDevice();
return device != null
&& ((device.getVendorId() == 0x20d6 && device.getProductId() == 0x2063)
|| (device.getVendorId() == 0x0e6f && device.getProductId() == 0x02f0))
&& device.getMotionRange(MotionEvent.AXIS_RX, event.getSource()) != null
&& device.getMotionRange(MotionEvent.AXIS_RY, event.getSource()) != null
&& device.getMotionRange(MotionEvent.AXIS_Z, event.getSource()) != null
&& device.getMotionRange(MotionEvent.AXIS_RZ, event.getSource()) != null;
}
public static float rightX(MotionEvent event, boolean wired) {
return axis(event, wired ? MotionEvent.AXIS_RX : MotionEvent.AXIS_Z);
}
public static float rightY(MotionEvent event, boolean wired) {
return axis(event, wired ? MotionEvent.AXIS_RY : MotionEvent.AXIS_RZ);
}
public static float leftTrigger(MotionEvent event, boolean wired) {
if (wired) return Math.max(0f, Math.min(1f, (event.getAxisValue(MotionEvent.AXIS_Z) + 1f) / 2f));
float value = event.getAxisValue(MotionEvent.AXIS_LTRIGGER);
return value != 0f ? value : event.getAxisValue(MotionEvent.AXIS_BRAKE);
}
public static float rightTrigger(MotionEvent event, boolean wired) {
if (wired) return Math.max(0f, Math.min(1f, (event.getAxisValue(MotionEvent.AXIS_RZ) + 1f) / 2f));
float value = event.getAxisValue(MotionEvent.AXIS_RTRIGGER);
return value != 0f ? value : event.getAxisValue(MotionEvent.AXIS_GAS);
}
private static float axis(MotionEvent event, int axis) {
float value = event.getAxisValue(axis);
return Math.abs(value) < DEAD_ZONE ? 0f : value;
@@ -0,0 +1,358 @@
package com.droiddeck.launcher.input
import android.os.SystemClock
import android.view.Choreographer
import android.view.KeyEvent
import android.view.MotionEvent
import kotlin.math.abs
import kotlin.math.hypot
class PadMouse(private val listener: Listener, private val clock: () -> Long = SystemClock::uptimeMillis) {
interface Listener {
fun active(): Boolean
fun onScrollStart()
fun onScroll(dx: Float, dy: Float)
fun onKey(evdev: Int, pressed: Boolean)
fun onKeyboard()
fun onNavigate()
fun scrollSpeed(): Float
}
private class Sticks {
var leftX = 0f
var leftY = 0f
var rightX = 0f
var rightY = 0f
var leftTrigger = 0f
var rightTrigger = 0f
fun clear() {
leftX = 0f
leftY = 0f
rightX = 0f
rightY = 0f
leftTrigger = 0f
rightTrigger = 0f
}
}
private val choreographer = Choreographer.getInstance()
private val physical = HashMap<Int, Sticks>()
private val touch = Sticks()
private val total = Sticks()
private val triggerAxes = HashMap<Int, Boolean>()
private val wiredLayout = HashMap<Int, Boolean>()
private val down = HashMap<Int, MutableSet<Int>>()
private val sent = HashMap<Int, IntArray>()
private val held = LinkedHashMap<Int, Int>()
private var scrolling = false
private var stickDirection = NONE
private var stickRepeatAt = 0L
private var frameAt = 0L
private var running = false
private val frame = object : Choreographer.FrameCallback {
override fun doFrame(frameTimeNanos: Long) {
if (!running) return
if (!listener.active()) {
reset()
return
}
val dt = if (frameAt == 0L) 1f / 60f else ((frameTimeNanos - frameAt) / 1e9f).coerceIn(0f, 0.05f)
frameAt = frameTimeNanos
step(dt)
if (moving()) choreographer.postFrameCallback(this) else stop()
}
}
fun onKey(event: KeyEvent): Boolean {
if (event.keyCode == KeyEvent.KEYCODE_BUTTON_L2 || event.keyCode == KeyEvent.KEYCODE_BUTTON_R2) {
if (event.repeatCount == 0 && !hasTriggerAxes(event)) {
val sticks = physical.getOrPut(event.deviceId) { Sticks() }
val value = if (event.action == KeyEvent.ACTION_DOWN) 1f else 0f
if (event.keyCode == KeyEvent.KEYCODE_BUTTON_L2) sticks.leftTrigger = value else sticks.rightTrigger = value
wake()
}
return true
}
val button = BUTTONS[event.keyCode] ?: return false
if (event.repeatCount > 0) return true
when (event.action) {
KeyEvent.ACTION_DOWN -> set(source(event.deviceId, SOURCE_KEYS), button, true)
KeyEvent.ACTION_UP -> set(source(event.deviceId, SOURCE_KEYS), button, false)
}
return true
}
fun onMotion(event: MotionEvent): Boolean {
if (event.actionMasked != MotionEvent.ACTION_MOVE) return true
val sticks = physical.getOrPut(event.deviceId) { Sticks() }
val wired = wiredLayout.getOrPut(event.deviceId) { PadBridge.wiredXboxLayout(event) }
sticks.leftX = event.getAxisValue(MotionEvent.AXIS_X)
sticks.leftY = event.getAxisValue(MotionEvent.AXIS_Y)
sticks.rightX = PadBridge.rightX(event, wired)
sticks.rightY = PadBridge.rightY(event, wired)
if (hasTriggerAxes(event)) {
sticks.leftTrigger = PadBridge.leftTrigger(event, wired)
sticks.rightTrigger = PadBridge.rightTrigger(event, wired)
}
val hatX = event.getAxisValue(MotionEvent.AXIS_HAT_X)
val hatY = event.getAxisValue(MotionEvent.AXIS_HAT_Y)
val hat = source(event.deviceId, SOURCE_HAT)
set(hat, UP, hatY < -0.5f)
set(hat, DOWN, hatY > 0.5f)
set(hat, LEFT, hatX < -0.5f)
set(hat, RIGHT, hatX > 0.5f)
wake()
return true
}
fun onTouchPad(state: PadState) {
touch.leftX = state.leftX
touch.leftY = state.leftY
touch.rightX = state.rightX
touch.rightY = state.rightY
touch.leftTrigger = state.leftTrigger
touch.rightTrigger = state.rightTrigger
for (button in TOUCH_BUTTONS) {
set(SOURCE_TOUCH, button, state.isDown(button))
}
set(SOURCE_TOUCH, UP, state.up)
set(SOURCE_TOUCH, DOWN, state.down)
set(SOURCE_TOUCH, LEFT, state.left)
set(SOURCE_TOUCH, RIGHT, state.right)
wake()
}
fun reset() {
stop()
physical.clear()
triggerAxes.clear()
wiredLayout.clear()
touch.clear()
down.clear()
sent.clear()
stickDirection = NONE
scrolling = false
held.keys.toList().reversed().forEach { code ->
held.remove(code)
send(code, false)
}
}
internal fun step(dt: Float) {
sum()
val lx = total.leftX
val ly = total.leftY
val held = heldDirection()
val direction = when {
held != NONE -> held
hypot(lx, ly) < STICK_NAVIGATE -> NONE
abs(lx) > abs(ly) -> if (lx > 0) RIGHT else LEFT
else -> if (ly > 0) DOWN else UP
}
val now = clock()
if (direction != stickDirection) {
stickDirection = direction
if (direction != NONE) {
tap(direction)
stickRepeatAt = now + REPEAT_FIRST_MS
}
} else if (direction != NONE && now >= stickRepeatAt) {
tap(direction)
stickRepeatAt = now + REPEAT_MS
}
val gain = gain(total.rightX, total.rightY)
val sx = total.rightX * gain
val triggers = trigger(total.rightTrigger) - trigger(total.leftTrigger)
val vertical = (total.rightY * gain + triggers).coerceIn(-1f, 1f)
if (sx != 0f || vertical != 0f) {
if (!scrolling) {
scrolling = true
listener.onScrollStart()
}
val speed = listener.scrollSpeed() * dt
listener.onScroll(sx * speed, vertical * speed)
} else if (scrolling) {
scrolling = false
press(KEY_SETTLE)
release(KEY_SETTLE)
}
}
private fun set(source: Int, button: Int, pressed: Boolean) {
val sources = down.getOrPut(button) { HashSet() }
val was = sources.isNotEmpty()
if (pressed) sources.add(source) else sources.remove(source)
val now = sources.isNotEmpty()
if (was == now) return
if (button in DIRECTIONS) {
if (now) step(0f)
wake()
return
}
if (now) {
if (button == PadState.Y) {
listener.onKeyboard()
return
}
val codes = codesFor(button)
sent[button] = codes
codes.forEach { press(it) }
} else {
sent.remove(button)?.reversed()?.forEach { release(it) }
}
}
private fun hasTriggerAxes(event: android.view.InputEvent): Boolean = triggerAxes.getOrPut(event.deviceId) {
val device = event.device ?: return@getOrPut true
device.getMotionRange(MotionEvent.AXIS_LTRIGGER) != null || device.getMotionRange(MotionEvent.AXIS_BRAKE) != null ||
(event is MotionEvent && wiredLayout.getOrPut(event.deviceId) { PadBridge.wiredXboxLayout(event) })
}
private fun heldDirection(): Int {
for (direction in DIRECTIONS) if (down[direction]?.isNotEmpty() == true) return direction
return NONE
}
private fun sum() {
total.leftX = touch.leftX
total.leftY = touch.leftY
total.rightX = touch.rightX
total.rightY = touch.rightY
total.leftTrigger = touch.leftTrigger
total.rightTrigger = touch.rightTrigger
for (sticks in physical.values) {
total.leftX += sticks.leftX
total.leftY += sticks.leftY
total.rightX += sticks.rightX
total.rightY += sticks.rightY
total.leftTrigger += sticks.leftTrigger
total.rightTrigger += sticks.rightTrigger
}
}
private fun tap(direction: Int) {
listener.onNavigate()
val codes = codesFor(direction)
codes.forEach { press(it) }
codes.reversed().forEach { release(it) }
}
private fun codesFor(button: Int): IntArray = when (button) {
PadState.A -> intArrayOf(KEY_ENTER)
PadState.X -> intArrayOf(KEY_MENU)
PadState.B -> intArrayOf(KEY_ESC, KEY_BACK)
PadState.START -> intArrayOf(KEY_ENTER)
PadState.LB -> intArrayOf(KEY_LEFTCTRL, KEY_PAGEUP)
PadState.RB -> intArrayOf(KEY_LEFTCTRL, KEY_PAGEDOWN)
UP -> intArrayOf(KEY_UP)
DOWN -> intArrayOf(KEY_DOWN)
LEFT -> intArrayOf(KEY_LEFT)
RIGHT -> intArrayOf(KEY_RIGHT)
else -> IntArray(0)
}
private fun wake() {
if (moving() && !running) {
running = true
frameAt = 0L
choreographer.postFrameCallback(frame)
}
}
private fun moving(): Boolean {
if (heldDirection() != NONE || stickDirection != NONE || scrolling) return true
sum()
return hypot(total.leftX, total.leftY) >= STICK_NAVIGATE || gain(total.rightX, total.rightY) != 0f ||
trigger(total.leftTrigger) != 0f || trigger(total.rightTrigger) != 0f
}
private fun stop() {
running = false
frameAt = 0L
choreographer.removeFrameCallback(frame)
}
private fun press(code: Int) {
val count = held[code] ?: 0
held[code] = count + 1
if (count == 0) send(code, true)
}
private fun release(code: Int) {
val count = held[code] ?: return
if (count > 1) {
held[code] = count - 1
return
}
held.remove(code)
send(code, false)
}
private fun send(code: Int, pressed: Boolean) {
listener.onKey(code, pressed)
}
companion object {
private const val DEAD_ZONE = 0.15f
private const val STICK_NAVIGATE = 0.5f
private const val REPEAT_FIRST_MS = 300L
private const val REPEAT_MS = 120L
private const val SOURCE_KEYS = 0
private const val SOURCE_HAT = 1
private const val SOURCE_TOUCH = Int.MIN_VALUE
private const val NONE = -1
private const val UP = 100
private const val DOWN = 101
private const val LEFT = 102
private const val RIGHT = 103
private val DIRECTIONS = intArrayOf(UP, DOWN, LEFT, RIGHT)
private val TOUCH_BUTTONS = intArrayOf(PadState.A, PadState.B, PadState.X, PadState.Y, PadState.LB, PadState.RB, PadState.START)
private const val KEY_ESC = 1
private const val KEY_ENTER = 28
private const val KEY_UP = 103
private const val KEY_LEFT = 105
private const val KEY_RIGHT = 106
private const val KEY_DOWN = 108
private const val KEY_BACK = 158
private const val KEY_LEFTCTRL = 29
private const val KEY_PAGEUP = 104
private const val KEY_PAGEDOWN = 109
private const val KEY_MENU = 127
private const val KEY_SETTLE = 183
private val BUTTONS = mapOf(
KeyEvent.KEYCODE_BUTTON_A to PadState.A,
KeyEvent.KEYCODE_BUTTON_B to PadState.B,
KeyEvent.KEYCODE_BUTTON_X to PadState.X,
KeyEvent.KEYCODE_BUTTON_Y to PadState.Y,
KeyEvent.KEYCODE_BUTTON_L1 to PadState.LB,
KeyEvent.KEYCODE_BUTTON_R1 to PadState.RB,
KeyEvent.KEYCODE_BUTTON_START to PadState.START,
KeyEvent.KEYCODE_DPAD_UP to UP,
KeyEvent.KEYCODE_DPAD_DOWN to DOWN,
KeyEvent.KEYCODE_DPAD_LEFT to LEFT,
KeyEvent.KEYCODE_DPAD_RIGHT to RIGHT,
)
internal fun curve(x: Float, y: Float): Pair<Float, Float> {
val gain = gain(x, y)
return x * gain to y * gain
}
private fun gain(x: Float, y: Float): Float {
val length = hypot(x, y)
if (length <= DEAD_ZONE) return 0f
val scaled = (length.coerceAtMost(1f) - DEAD_ZONE) / (1f - DEAD_ZONE)
return scaled * scaled / length
}
private fun source(deviceId: Int, kind: Int): Int = deviceId * 2 + kind
private fun trigger(value: Float): Float =
if (value <= DEAD_ZONE) 0f else (value.coerceAtMost(1f) - DEAD_ZONE) / (1f - DEAD_ZONE)
}
}
@@ -0,0 +1,190 @@
package com.droiddeck.launcher.input
import android.os.Handler
import android.os.Looper
import android.view.Choreographer
import android.view.MotionEvent
import android.view.VelocityTracker
import kotlin.math.abs
import kotlin.math.exp
import kotlin.math.floor
import kotlin.math.hypot
import kotlin.math.pow
class ScrollTouchGestures(
private val slopPx: Float,
private val minFlingPx: Float,
private val listener: Listener,
) {
interface Listener {
fun onMove(x: Float, y: Float)
fun onButton(button: Int, pressed: Boolean)
fun onScroll(dx: Float, dy: Float)
fun onLongPress()
}
private val handler = Handler(Looper.getMainLooper())
private val choreographer = Choreographer.getInstance()
private var tracker: VelocityTracker? = null
private var pointerId = -1
private var startX = 0f
private var startY = 0f
private var lastX = 0f
private var lastY = 0f
private var movedAt = 0L
private var scrolling = false
private var vertical = true
private var held = false
private var dragging = false
private var stoppedFling = false
private var clicked = 0
private var flingX = 0f
private var flingY = 0f
private var flingAt = 0L
private var flinging = false
private val longPress = Runnable {
held = true
listener.onLongPress()
}
private val clickUp = Runnable { releaseClick() }
private val flingFrame = object : Choreographer.FrameCallback {
override fun doFrame(frameTimeNanos: Long) {
if (!flinging) return
val dt = if (flingAt == 0L) 1f / 60f else ((frameTimeNanos - flingAt) / 1e9f).coerceIn(0f, 0.05f)
flingAt = frameTimeNanos
val decay = exp(-dt / FLING_TIME_S)
listener.onScroll(flingX * dt, flingY * dt)
flingX *= decay
flingY *= decay
if (hypot(flingX, flingY) < FLING_STOP_PX) flinging = false else choreographer.postFrameCallback(this)
}
}
fun cancel() {
handler.removeCallbacks(longPress)
stopFling()
releaseClick()
if (dragging) listener.onButton(PointerGestures.BTN_LEFT, false)
dragging = false
held = false
scrolling = false
pointerId = -1
tracker?.recycle()
tracker = null
}
fun onTouch(event: MotionEvent): Boolean {
when (event.actionMasked) {
MotionEvent.ACTION_DOWN -> {
stoppedFling = flinging
cancel()
pointerId = event.getPointerId(0)
startX = event.x
startY = event.y
lastX = startX
lastY = startY
tracker = VelocityTracker.obtain().also { it.addMovement(event) }
listener.onMove(startX, startY)
handler.postDelayed(longPress, HOLD_MS)
}
MotionEvent.ACTION_MOVE -> {
val index = event.findPointerIndex(pointerId).takeIf { it >= 0 } ?: return true
tracker?.addMovement(event)
val x = event.getX(index)
val y = event.getY(index)
if (!scrolling && !dragging && hypot(x - startX, y - startY) > slopPx) {
handler.removeCallbacks(longPress)
if (held) {
dragging = true
listener.onButton(PointerGestures.BTN_LEFT, true)
} else {
scrolling = true
vertical = abs(y - startY) >= abs(x - startX)
lastX = startX
lastY = startY
}
}
if (dragging) listener.onMove(x, y)
follow(event, x, y)
}
MotionEvent.ACTION_POINTER_UP -> {
if (event.getPointerId(event.actionIndex) == pointerId) cancel()
}
MotionEvent.ACTION_UP -> {
handler.removeCallbacks(longPress)
if (pointerId == -1 || event.getPointerId(0) != pointerId) return true
tracker?.addMovement(event)
follow(event, event.x, event.y)
when {
scrolling -> tracker?.takeIf { event.eventTime - movedAt <= FLING_STILL_MS }?.let { t ->
t.computeCurrentVelocity(1000)
val vx = if (vertical) 0f else -t.getXVelocity(pointerId)
val vy = if (vertical) -t.getYVelocity(pointerId) else 0f
if (hypot(vx, vy) >= minFlingPx) startFling(vx, vy)
}
dragging -> listener.onButton(PointerGestures.BTN_LEFT, false)
held -> click(PointerGestures.BTN_RIGHT)
!stoppedFling -> click(PointerGestures.BTN_LEFT)
}
dragging = false
held = false
scrolling = false
pointerId = -1
tracker?.recycle()
tracker = null
}
MotionEvent.ACTION_CANCEL -> cancel()
}
return true
}
private fun follow(event: MotionEvent, x: Float, y: Float) {
if (!scrolling || (x == lastX && y == lastY)) return
movedAt = event.eventTime
if (vertical) listener.onScroll(0f, lastY - y) else listener.onScroll(lastX - x, 0f)
lastX = x
lastY = y
}
private fun click(button: Int) {
releaseClick()
listener.onButton(button, true)
clicked = button
handler.postDelayed(clickUp, CLICK_MS)
}
private fun releaseClick() {
handler.removeCallbacks(clickUp)
if (clicked != 0) listener.onButton(clicked, false)
clicked = 0
}
private fun startFling(vx: Float, vy: Float) {
flingX = vx
flingY = vy
flingAt = 0L
flinging = true
choreographer.postFrameCallback(flingFrame)
}
private fun stopFling() {
flinging = false
choreographer.removeFrameCallback(flingFrame)
}
companion object {
fun wheelUnits(pixels: Int, viewSize: Int): Int {
if (pixels == 0) return 0
val step = floor(viewSize.toDouble().pow(2.0 / 3.0)).coerceAtLeast(1.0)
val units = (abs(pixels) * 120.0 / step).toInt() + 1
return if (pixels < 0) -units else units
}
private const val HOLD_MS = 500L
private const val CLICK_MS = 50L
private const val FLING_TIME_S = 0.45f
private const val FLING_STOP_PX = 40f
private const val FLING_STILL_MS = 60L
}
}
@@ -136,7 +136,7 @@ object SteamTouchConfig {
null
}
fun steamRoot(context: Context) = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam")
fun steamRoot(context: Context) = com.droiddeck.launcher.session.SessionPrefs.activeSteamRoot(context)
fun configDir(context: Context): File? {
val base = File(steamRoot(context), "steamapps/common/Steam Controller Configs")
@@ -22,7 +22,11 @@ object DeckyManager {
private const val RELEASES = "https://api.github.com/repos/$REPOSITORY/releases?per_page=30"
private const val PATH = "root/homebrew/services/PluginLoader"
private const val VERSION = "root/homebrew/services/.droiddeck-decky-version"
private const val CEF_REMOTE_DEBUG_MARKER = "root/.local/share/Steam/.cef-enable-remote-debugging"
// droiddeck-session reads the marker from the root of the client it runs.
private val CEF_REMOTE_DEBUG_MARKERS = listOf(
"root/.local/share/Steam/.cef-enable-remote-debugging",
"root/.droiddeck-x64/.local/share/Steam/.cef-enable-remote-debugging",
)
@Volatile var downloadingBinary = false
private set
@@ -39,29 +43,34 @@ object DeckyManager {
fun loader(context: Context) = File(LinuxRuntime.rootDir(context), PATH)
fun supervisorEnabled(context: Context) = File(LinuxRuntime.rootDir(context), "root/.droiddeck-decky-enabled").isFile
private fun cefMarkers(context: Context, creating: Boolean = false) =
CEF_REMOTE_DEBUG_MARKERS.map { File(LinuxRuntime.rootDir(context), it) }.filter { marker ->
!creating || !marker.path.contains("/.droiddeck-x64/") ||
com.droiddeck.launcher.storage.SteamStorage.x64ClientPresent(marker.parentFile ?: marker)
}
fun syncCefMarker(context: Context) {
val marker = File(LinuxRuntime.rootDir(context), CEF_REMOTE_DEBUG_MARKER)
if (!supervisorEnabled(context)) {
marker.delete()
cefMarkers(context).forEach { it.delete() }
return
}
runCatching {
marker.parentFile?.mkdirs()
marker.createNewFile()
for (marker in cefMarkers(context, creating = true)) {
runCatching {
marker.parentFile?.mkdirs()
marker.createNewFile()
}
}
}
fun setSupervisorEnabled(context: Context, enabled: Boolean) {
val marker = File(LinuxRuntime.rootDir(context), "root/.droiddeck-decky-enabled")
val cefMarker = File(LinuxRuntime.rootDir(context), CEF_REMOTE_DEBUG_MARKER)
if (enabled) {
marker.parentFile?.mkdirs()
marker.writeText("enabled\n")
cefMarker.parentFile?.mkdirs()
cefMarker.createNewFile()
cefMarkers(context, creating = true).forEach { it.parentFile?.mkdirs(); it.createNewFile() }
} else {
marker.delete()
cefMarker.delete()
cefMarkers(context).forEach { it.delete() }
}
}
@@ -120,11 +129,12 @@ object DeckyManager {
if (!validElf(temp, release.machine)) { temp.delete(); return context.getString(R.string.deckymgr_not_elf) }
if (SessionState.running) { temp.delete(); return context.getString(R.string.deckymgr_session_started_loader) }
if (!temp.setExecutable(true, false)) { temp.delete(); return context.getString(R.string.deckymgr_loader_not_executable) }
val cefMarker = File(LinuxRuntime.rootDir(context), CEF_REMOTE_DEBUG_MARKER)
val cefReady = !supervisorEnabled(context) || cefMarker.isFile || runCatching {
cefMarker.parentFile?.mkdirs()
cefMarker.createNewFile() || cefMarker.isFile
}.getOrDefault(false)
val cefReady = !supervisorEnabled(context) || cefMarkers(context, creating = true).all { cefMarker ->
cefMarker.isFile || runCatching {
cefMarker.parentFile?.mkdirs()
cefMarker.createNewFile() || cefMarker.isFile
}.getOrDefault(false)
}
if (!cefReady) { temp.delete(); return context.getString(R.string.deckymgr_cef_failed) }
val staged = File(target.parentFile, "PluginLoader.new")
staged.delete()
@@ -140,7 +150,7 @@ object DeckyManager {
old.delete()
File(target.parentFile, ".droiddeck-decky-version").writeText(release.tag + "\n")
File(target.parentFile, ".droiddeck-decky-arch").writeText(if (release.machine == 183) "arm64\n" else "x86_64\n")
if (!supervisorEnabled(context)) cefMarker.delete()
if (!supervisorEnabled(context)) cefMarkers(context).forEach { it.delete() }
return null
}
@@ -272,7 +282,7 @@ object DeckyManager {
File(root, PATH).delete()
File(root, VERSION).delete()
File(root, "root/homebrew/services/.droiddeck-decky-arch").delete()
File(root, CEF_REMOTE_DEBUG_MARKER).delete()
CEF_REMOTE_DEBUG_MARKERS.forEach { File(root, it).delete() }
if (wipeData) File(root, "root/homebrew").deleteRecursively()
}
@@ -12,7 +12,7 @@ import java.nio.file.attribute.BasicFileAttributes;
import java.util.function.LongConsumer;
import java.util.function.Predicate;
final class RuntimeFileTree {
public final class RuntimeFileTree {
private RuntimeFileTree() {}
static volatile Predicate<Path> refuseForTest;
@@ -35,7 +35,7 @@ final class RuntimeFileTree {
if (progress != null) progress.accept(++removed[0]);
}
static boolean deleteWhatCan(File root, LongConsumer progress) {
public static boolean deleteWhatCan(File root, LongConsumer progress) {
return deleteWhatCan(root.toPath(), progress, new long[]{0});
}
@@ -0,0 +1,71 @@
package com.droiddeck.launcher.session
import android.content.Context
import android.util.Log
import com.droiddeck.launcher.core.DownloadSource
import com.droiddeck.launcher.core.Downloader
import org.json.JSONObject
import java.io.File
/**
* The FEX build DroidDeck-Components' fex.json names for the x86-64 client. A session uses the last copy read.
* Components' build-fex.yml replaces the archive on its single `fex` release and records the build's version;
* a release per build (fex-2610-r5) is still read.
*/
object FexCatalog {
private const val TAG = "FexCatalog"
const val CATALOG_URL = "https://raw.githubusercontent.com/Droid-Deck/DroidDeck-Components/main/fex.json"
private const val RELEASES = "https://github.com/Droid-Deck/DroidDeck-Components/releases/download/"
private const val REFRESH_MS = 10 * 60 * 1000L
private val TAG_FORMAT = Regex("""fex-\d+-r\d+""")
private val VERSION_FORMAT = Regex("""\d+-r\d+""")
private val SHA256 = Regex("[0-9a-f]{64}")
data class Build(val tag: String, val url: String, val sha256: String)
@Volatile private var refreshedAt = 0L
private fun file(context: Context) = File(context.filesDir, "fex-catalog.json")
internal fun parse(text: String): Build? = runCatching {
val o = JSONObject(text)
val release = o.getString("release")
val url = o.getString("url")
val sha = o.getString("sha256").lowercase()
val version = o.optString("version")
val tag = when {
TAG_FORMAT.matches(release) -> release
release == "fex" && VERSION_FORMAT.matches(version) -> "fex-$version"
else -> return@runCatching null
}
Build(tag, url, sha).takeIf { url == "$RELEASES$release/fex.tzst" && SHA256.matches(sha) }
}.getOrNull()
fun cached(context: Context): Build? = runCatching { parse(file(context).readText()) }.getOrNull()
/** Reads fex.json straight from GitHub (a digest never comes from a mirror), at most every ten minutes. */
@Synchronized
fun refresh(context: Context, now: Long = System.currentTimeMillis()): Build? {
if (now - refreshedAt < REFRESH_MS) return cached(context)
refreshedAt = now
val text = Downloader.downloadDirect(CATALOG_URL) ?: return cached(context)
val build = parse(text) ?: run {
Log.w(TAG, "fex.json not usable; keeping the last one")
return cached(context)
}
runCatching {
val f = file(context)
val tmp = File(f.path + ".tmp")
tmp.writeText(text)
if (!tmp.renameTo(f)) tmp.delete()
}
return build
}
/** For droiddeck-steam-x64: the build and every source its archive may come from (it checks the sha256). */
fun environment(build: Build, sources: List<String> = DownloadSource.candidates(build.url)): List<String> = listOf(
"BL_FEX_TAG=${build.tag}",
"BL_FEX_SHA256=${build.sha256}",
"BL_FEX_URLS=" + sources.joinToString(" "),
)
}
@@ -64,6 +64,13 @@ object GameEnvironmentStore {
return config
}
@Synchronized
fun setGameSync(context: Context, scope: String, backend: String?): GameEnvironment.Config {
val config = GameEnvironment.withGameSync(read(context), scope, backend)
save(context, config)
return config
}
@Synchronized
fun setGameNoSteam(context: Context, scope: String, on: Boolean): GameEnvironment.Config {
val config = GameEnvironment.withGameNoSteam(read(context), scope, on)
@@ -50,6 +50,11 @@ object GameStorage {
}
}
const val X64_CARD_DIR = "x86-64"
const val X64_CARD_GUEST = "/mnt/droiddeck-sd/$X64_CARD_DIR"
fun x64CardLibrary(context: Context): File? = effective(context)?.let { File(it.path, X64_CARD_DIR) }
class Library(val host: File, val guest: String, val label: String)
fun gamesFolderLibraries(context: Context): List<Library> {
@@ -0,0 +1,62 @@
package com.droiddeck.launcher.session
import android.content.Context
import com.droiddeck.launcher.core.GameEnvironment
import com.droiddeck.launcher.frontend.AddedGames
import com.droiddeck.launcher.frontend.Library
import com.droiddeck.launcher.stores.epic.EpicClient
object NoSteamLaunch {
const val PROTON_RUN = "/usr/local/bin/droiddeck-proton-run"
const val EPIC = "/usr/local/bin/droiddeck-epic"
class Command(val program: String, val args: List<String>, val name: String)
fun scope(game: Library.SteamGame): String? = when {
game.library == Library.EPIC_LAUNCHER -> EpicClient.shortcutAppId.toString()
game.library == Library.ADDED || game.appId > 0 -> game.profileKey
else -> null
}?.takeIf { it.isNotEmpty() && GameEnvironment.validScope(it) }
fun enabled(config: GameEnvironment.Config, game: Library.SteamGame): Boolean =
scope(game)?.let { GameEnvironment.gameNoSteam(config, it) } == true
fun enabled(context: Context, game: Library.SteamGame): Boolean =
scope(game) != null && runCatching { enabled(GameEnvironmentStore.read(context), game) }.getOrDefault(false)
fun enabled(context: Context, appId: Long): Boolean = runCatching {
GameEnvironment.gameNoSteam(GameEnvironmentStore.read(context), (appId and 0xFFFFFFFFL).toString())
}.getOrDefault(false)
fun command(game: Library.SteamGame, added: AddedGames.Game?): Command? = when {
game.library == Library.EPIC_LAUNCHER -> Command(EPIC, listOf("run"), game.name)
game.library == Library.ADDED -> added?.let { a -> a.steamAppId?.let { Command(PROTON_RUN, listOf("--app", it.toString()), a.name) } ?: addedCommand(a) }
game.appId > 0 -> Command(PROTON_RUN, listOf("--app", game.appId.toString()), game.name)
else -> null
}
fun addedCommand(game: AddedGames.Game): Command = Command(
PROTON_RUN,
listOf("--app", (game.appId and 0xFFFFFFFFL).toString(), "--dir", game.guestDir, "--options", game.launchOptions, "--", game.guestExe),
game.name,
)
fun settingsId(game: AddedGames.Game): Long = (game.steamAppId?.toLong() ?: game.appId) and 0xFFFFFFFFL
fun addedGame(context: Context, game: Library.SteamGame): AddedGames.Game? {
val id = game.appId.toLong() and 0xFFFFFFFFL
return AddedGames.scan(context).firstOrNull { settingsId(it) == id }
}
fun nameFor(program: String?, args: List<String>): String? = when (program) {
EPIC -> Library.EPIC_LAUNCHER_NAME
PROTON_RUN -> args.getOrNull(args.indexOf("--app") + 1)?.takeIf { args.contains("--app") }?.let { names[it] }
else -> null
}
val names = java.util.concurrent.ConcurrentHashMap<String, String>()
fun remember(command: Command) {
if (command.program == PROTON_RUN) command.args.getOrNull(command.args.indexOf("--app") + 1)?.let { names[it] = command.name }
}
}
@@ -17,8 +17,13 @@ object OfflineMode {
private fun loginUsers(root: File) = File(root, "root/.local/share/Steam/config/loginusers.vdf")
private fun x64LoginUsers(root: File) = File(root, "root/.droiddeck-x64/.local/share/Steam/config/loginusers.vdf")
fun account(context: Context): String? {
val file = loginUsers(LinuxRuntime.rootDir(context))
val root = LinuxRuntime.rootDir(context)
// Until its first start the x86-64 client has no sign-in; droiddeck-steam-x64 --sync-login copies the arm64 one.
val file = x64LoginUsers(root).takeIf { SessionPrefs.activeSteamClientArch(context) == "x86_64" && it.isFile }
?: loginUsers(root)
val text = try {
if (file.isFile) file.readText() else return null
} catch (e: Exception) {
@@ -36,7 +41,11 @@ object OfflineMode {
}
fun apply(context: Context, root: File) {
val file = loginUsers(root)
applyTo(context, loginUsers(root))
applyTo(context, x64LoginUsers(root))
}
private fun applyTo(context: Context, file: File) {
if (!file.isFile) return
val want = if (enabled(context)) "1" else "0"
try {
@@ -7,26 +7,21 @@ import android.content.Context
import android.os.Build
import android.provider.Settings
import android.util.Log
import android.view.Display
object PerfMode {
private const val TAG = "PerfMode"
fun apply(a: Activity, fpsLimit: Int = 0): String {
fun apply(a: Activity, fpsLimit: Int = 0, refreshLimit: Int = 0): String {
val parts = ArrayList<String>(2)
parts += try {
val display = if (Build.VERSION.SDK_INT >= 30) a.display else @Suppress("DEPRECATION") a.windowManager.defaultDisplay
if (display == null || Build.VERSION.SDK_INT < 23) "display mode unchanged" else {
val cur = display.mode
val cap = peakRefreshCap(a)
val sameSize = display.supportedModes
.filter { it.physicalWidth == cur.physicalWidth && it.physicalHeight == cur.physicalHeight }
.filter { it.refreshRate <= cap + 0.5f }
.ifEmpty { display.supportedModes.filter { it.modeId == cur.modeId } }
val best = sameSize.filter { cadenceFits(it.refreshRate, fpsLimit) }.maxByOrNull { it.refreshRate }
?: sameSize.maxByOrNull { it.refreshRate }
if (best != null && best.modeId != cur.modeId &&
(best.refreshRate > cur.refreshRate + 0.5f || !cadenceFits(cur.refreshRate, fpsLimit))) {
val cap = minOf(peakRefreshCap(a), refreshLimit.takeIf { it > 0 }?.toFloat() ?: Float.POSITIVE_INFINITY)
val best = selectMode(display.supportedModes, cur, fpsLimit, cap)
if (best != null && a.window.attributes.preferredDisplayModeId != best.modeId) {
a.window.attributes = a.window.attributes.apply { preferredDisplayModeId = best.modeId }
"display ${cur.refreshRate.toInt()} → ${best.refreshRate.toInt()} Hz (mode ${best.modeId})"
} else "display ${cur.refreshRate.toInt()} Hz"
@@ -59,6 +54,16 @@ object PerfMode {
return if (peak >= 1f) peak else Float.POSITIVE_INFINITY
}
internal fun selectMode(modes: Array<Display.Mode>, current: Display.Mode, fpsLimit: Int, cap: Float): Display.Mode? {
val sameSize = modes.filter {
it.physicalWidth == current.physicalWidth && it.physicalHeight == current.physicalHeight
}
val allowed = sameSize.filter { it.refreshRate <= cap + 0.5f }
return allowed.filter { cadenceFits(it.refreshRate, fpsLimit) }.maxByOrNull { it.refreshRate }
?: allowed.maxByOrNull { it.refreshRate }
?: sameSize.minByOrNull { it.refreshRate }
}
fun cadenceFits(hz: Float, fpsLimit: Int): Boolean {
if (fpsLimit <= 0) return true
val ratio = hz / fpsLimit
@@ -16,6 +16,7 @@ import android.view.InputDevice
import androidx.annotation.RequiresApi
import com.droiddeck.launcher.core.SessionPart
import com.droiddeck.launcher.input.ControllerPrefs
import com.droiddeck.launcher.input.JoyCon
import com.droiddeck.launcher.input.PadBridge
import java.io.DataInputStream
import kotlin.math.max
@@ -26,6 +27,7 @@ class RumbleComponent : SessionPart() {
private val playing = HashMap<Int, Motors>()
private val effects = HashMap<Int, Effect>()
private val controllers = HashMap<Int, Motors?>()
private val joyConPairs = HashMap<Long, Motors>()
private var preferences: SharedPreferences? = null
private var enabled = false
private var motorThread: HandlerThread? = null
@@ -78,6 +80,7 @@ class RumbleComponent : SessionPart() {
motorThread = null
phone = null
controllers.clear()
joyConPairs.clear()
val s = server
server = null
try { s?.close() } catch (e: Exception) { }
@@ -125,10 +128,24 @@ class RumbleComponent : SessionPart() {
}
private fun target(slot: Int): Motors? {
JoyCon.vrRumbleDevice(slot)?.let { hand -> return if (hand == PadBridge.NO_CONTROLLER) null else motors(hand) }
val left = PadBridge.joyConForSlot(slot, JoyCon.LEFT)
val right = PadBridge.joyConForSlot(slot, JoyCon.RIGHT)
if (left != PadBridge.NO_CONTROLLER && right != PadBridge.NO_CONTROLLER) {
val key = (left.toLong() shl 32) or (right.toLong() and 0xFFFFFFFFL)
joyConPairs[key]?.let { return it }
val l = motors(left)
val r = motors(right)
if (l != null && r != null) return JoyConPairMotors(l, r).also { joyConPairs[key] = it }
}
val id = PadBridge.controllerForSlot(slot)
if (id == PadBridge.NO_CONTROLLER) return phone
return motors(id) ?: phone
}
private fun motors(id: Int): Motors? {
if (!controllers.containsKey(id)) controllers[id] = try { controllerMotors(id) } catch (e: Exception) { null }
return controllers[id] ?: phone
return controllers[id]
}
private fun cancelAll() {
@@ -201,6 +218,25 @@ class RumbleComponent : SessionPart() {
}
}
@RequiresApi(31)
private class JoyConMotors(override val name: String, private val manager: VibratorManager) : Motors {
override fun play(strong: Int, weak: Int, ms: Long) =
manager.vibrate(CombinedVibration.createParallel(oneShot(manager.defaultVibrator, max(strong, weak), ms)))
override fun cancel() = manager.cancel()
}
private class JoyConPairMotors(private val left: Motors, private val right: Motors) : Motors {
override val name = "${left.name} and ${right.name}"
override fun play(strong: Int, weak: Int, ms: Long) {
if (strong == 0) left.cancel() else left.play(strong, 0, ms)
if (weak == 0) right.cancel() else right.play(0, weak, ms)
}
override fun cancel() {
left.cancel()
right.cancel()
}
}
private class LegacyPadMotors(override val name: String, private val vibrator: Vibrator) : Motors {
override fun play(strong: Int, weak: Int, ms: Long) = vibrator.vibrate(oneShot(vibrator, max(strong, weak), ms))
override fun cancel() {
@@ -230,7 +266,8 @@ class RumbleComponent : SessionPart() {
if (Build.VERSION.SDK_INT >= 31) {
val manager = device.vibratorManager
val ids = manager.vibratorIds.sortedArray()
return if (ids.isEmpty()) null else PadMotors(name, manager, ids)
if (ids.isEmpty()) return null
return if (JoyCon.side(device) >= 0) JoyConMotors(name, manager) else PadMotors(name, manager, ids)
}
@Suppress("DEPRECATION")
val vibrator = device.vibrator
@@ -21,7 +21,45 @@ object SessionArtifacts {
const val SCRUBBED_TREE_MARKER = ".scrubbed-tree"
fun collect(context: Context, dir: File, reason: String) {
private val collectingLock = java.util.concurrent.locks.ReentrantLock()
private val collectingDone = collectingLock.newCondition()
private val collecting = HashSet<File>()
/** Recovery leaves [dir] alone from the stop until [collect] has finished it. */
fun collectionStarted(dir: File) {
collectingLock.lock()
try { collecting.add(dir) } finally { collectingLock.unlock() }
}
private fun collectionDone(dir: File) {
collectingLock.lock()
try { collecting.remove(dir); collectingDone.signalAll() } finally { collectingLock.unlock() }
}
private fun inFlight(dir: File): Boolean {
collectingLock.lock()
try { return dir in collecting } finally { collectingLock.unlock() }
}
fun awaitCollections(timeoutMs: Long = 30_000) {
collectingLock.lock()
try {
var left = java.util.concurrent.TimeUnit.MILLISECONDS.toNanos(timeoutMs)
while (collecting.isNotEmpty() && left > 0) left = collectingDone.awaitNanos(left)
} finally {
collectingLock.unlock()
}
}
fun collect(context: Context, dir: File, reason: String, steamArch: String? = null) {
try {
collectNow(context, dir, reason, steamArch)
} finally {
collectionDone(dir)
}
}
private fun collectNow(context: Context, dir: File, reason: String, steamArch: String?) {
try {
val wayland = File(dir, "wayland.log")
if (!wayland.exists()) {
@@ -31,7 +69,8 @@ object SessionArtifacts {
}
LogRedactor.learnFromRuntime(LinuxRuntime.rootDir(context))
val record = Record(dir)
copySteamLogs(context, dir, record)
copySteamLogs(context, dir, record, steamArch)
VacDiagnostics.collect(context, dir)
SessionLogCapture.dumpCrashBuffer(File(dir, "crash.log"))
markScrubbed(scrubTree(record))
SessionEvents.record("session.artifacts_collected", mapOf("reason" to reason), dir)
@@ -149,8 +188,18 @@ object SessionArtifacts {
return record
}
private fun copySteamLogs(context: Context, dir: File, record: Record) {
val logs = File(LinuxRuntime.rootDir(context), "root/.local/share/Steam/logs")
private const val CLIENT_FILE = "steam-client.txt"
fun recordClient(dir: File, arch: String) {
runCatching { File(dir, CLIENT_FILE).writeText("$arch\n") }
}
/** The Steam client a session ran, when it recorded one. */
internal fun recordedClient(dir: File): String? =
runCatching { File(dir, CLIENT_FILE).readText().trim() }.getOrNull()?.takeIf { it == "arm64" || it == "x86_64" }
private fun copySteamLogs(context: Context, dir: File, record: Record, steamArch: String? = null) {
val logs = File(steamArch?.let { SessionPrefs.steamRoot(context, it) } ?: SessionPrefs.activeSteamRoot(context), "logs")
if (!logs.isDirectory) return
val out = File(dir, "steam").apply { mkdirs() }
logs.listFiles { f -> f.isFile && f.length() < 8L * 1024 * 1024 }?.forEach { src ->
@@ -171,7 +220,7 @@ object SessionArtifacts {
fun finishAbandoned(context: Context) {
val parent = LinuxRuntime.logDir(context)
val abandoned = parent.listFiles { f ->
SessionPaths.isSessionFolder(f) && !File(f, COMPLETE_MARKER).exists()
SessionPaths.isSessionFolder(f) && !File(f, COMPLETE_MARKER).exists() && !inFlight(f)
}?.sortedWith(SessionPaths.chronological) ?: return
if (abandoned.isEmpty()) return
val current = SessionPaths.current()
@@ -191,7 +240,10 @@ object SessionArtifacts {
"session died of a crash, the entry is in there unless the device rebooted.\n"
)
val record = Record(dir)
if (newest) copySteamLogs(context, dir, record)
if (newest) {
copySteamLogs(context, dir, record, recordedClient(dir))
VacDiagnostics.collect(context, dir)
}
SessionLogCapture.dumpCrashBuffer(File(dir, "crash.log"))
SessionEvents.record("session.artifacts_recovered", mapOf("newest" to newest), dir)
markScrubbed(scrubTree(record))
@@ -31,6 +31,7 @@ object SessionEvents {
SessionState.programArgs = emptyList()
SessionState.steamUi = null
SessionState.steamUrl = null
SessionState.steamVr = false
SessionState.firstFrameSeen = false
SessionState.suspended = false
}
@@ -67,9 +67,15 @@ object SessionFiles {
"usr/local/bin/droiddeck-steam-shim" to "usr/local/bin/droiddeck-steam-shim",
"usr/local/bin/droiddeck-steam-shortcuts" to "usr/local/bin/droiddeck-steam-shortcuts",
"usr/local/bin/droiddeck-store-launch" to "usr/local/bin/droiddeck-store-launch",
"usr/local/bin/droiddeck-epic" to "usr/local/bin/droiddeck-epic",
"usr/local/bin/droiddeck-epic-steam" to "usr/local/bin/droiddeck-epic-steam",
"usr/local/bin/droiddeck-open-url" to "usr/local/bin/droiddeck-open-url",
"usr/local/bin/droiddeck-steam-games" to "usr/local/bin/droiddeck-steam-games",
"usr/local/bin/droiddeck-pad-defaults" to "usr/local/bin/droiddeck-pad-defaults",
"usr/local/bin/droiddeck-steam-x64" to "usr/local/bin/droiddeck-steam-x64",
"usr/local/bin/droiddeck-x64-arm-proton" to "usr/local/bin/droiddeck-x64-arm-proton",
"usr/local/bin/droiddeck-x64-arm-webhelper" to "usr/local/bin/droiddeck-x64-arm-webhelper",
"usr/local/bin/droiddeck-x64-webhelper" to "usr/local/bin/droiddeck-x64-webhelper",
"usr/local/bin/droiddeck-steamvr" to "usr/local/bin/droiddeck-steamvr",
"usr/local/bin/droiddeck-steamvr-settings" to "usr/local/bin/droiddeck-steamvr-settings",
"usr/local/bin/droiddeck-steamvr-game" to "usr/local/bin/droiddeck-steamvr-game",
@@ -104,19 +110,24 @@ object SessionFiles {
val msitools = listOf("msiinfo", "cabextract", "7z", "7z.so", "libmsi-1.0.so.0", "libgsf-1.so.114", "libgcab-1.0.so.0")
.map { "usr/local/lib/droiddeck-msitools/$it" } +
listOf("NOTICE", "GPL-2", "GPL-3", "LGPL-2.1", "7zip-License", "7zip-unRarLicense").map { "usr/local/share/licenses/droiddeck-msitools/$it" }
val vr = listOf(
val handModels = listOf("left", "right").flatMap { side ->
listOf("droiddeck_hand_$side.json", "hand.mtl", "hand.png", "hand_open.obj", "hand_fist.obj", "thumb_index_open.obj",
"thumb_index_pinch.obj")
.map { "droiddeck/resources/rendermodels/droiddeck_hand_$side/$it" }
}
val vr = (listOf(
"droiddeck/bin/linuxarm64/driver_droiddeck.so",
"droiddeck/bin/linuxarm64/xr-display-mode",
"droiddeck/driver.vrdrivermanifest",
"droiddeck/resources/settings/default.vrsettings",
"layer/libVkLayer_droiddeck_vr.so",
"layer/VK_LAYER_DROIDDECK_vr.json",
).map { "usr/local/lib/droiddeck-vr/$it" to "usr/local/lib/droiddeck-vr/$it" } +
) + handModels).map { "usr/local/lib/droiddeck-vr/$it" to "usr/local/lib/droiddeck-vr/$it" } +
listOf("usr/local/lib/droiddeck-vr/layer/VK_LAYER_DROIDDECK_vr_implicit.json" to
"usr/share/vulkan/implicit_layer.d/droiddeck_vr.json")
val fexPreloads = listOf("x86_64", "i386").flatMap { arch ->
listOf("libblsession.so", "libfakeinput.so", "libfaultreport.so").map { "$arch/$it" to "usr/local/lib/droiddeck-fex/$arch/$it" }
} + listOf("libthunkaudit.so", "libvulkan-thunk.so").map { "x86_64/$it" to "usr/local/lib/droiddeck-fex/x86_64/$it" }
} + listOf("libthunkaudit.so", "libvulkan-thunk.so", "libwebkitinput.so").map { "x86_64/$it" to "usr/local/lib/droiddeck-fex/x86_64/$it" }
val optional = (arrayOf(
"usr/local/bin/gamescope" to "usr/local/bin/gamescope",
"usr/local/lib/droiddeck/uruntime" to "usr/local/lib/droiddeck/uruntime",
@@ -187,13 +198,18 @@ object SessionFiles {
Log.e(TAG, "could not write steam-compatibility-label")
}
val startupMovieDir = File(root, "root/.local/share/Steam/config/uioverrides/movies")
val startupMovie = BootMovie.prepare(context)
val bigPictureMovieInstalled = startupMovie != null &&
stageStartupMovie(startupMovie, startupMovieDir, "bigpicture_startup.webm")
if (startupMovie != null) stageStartupMovie(startupMovie, startupMovieDir, "steam_os_startup.webm")
if (bigPictureMovieInstalled) {
ensureStartupMovieDefault(File(root, "root/.local/share/Steam/config/config.vdf"))
// The x86-64 client's config is left alone until its first start creates it.
val steamRoots = listOf(File(root, "root/.local/share/Steam"), File(root, "root/.droiddeck-x64/.local/share/Steam"))
.filterIndexed { i, steam -> i == 0 || File(steam, "config").isDirectory }
for (steam in steamRoots) {
val startupMovieDir = File(steam, "config/uioverrides/movies")
val bigPictureMovieInstalled = startupMovie != null &&
stageStartupMovie(startupMovie, startupMovieDir, "bigpicture_startup.webm")
if (startupMovie != null) stageStartupMovie(startupMovie, startupMovieDir, "steam_os_startup.webm")
if (bigPictureMovieInstalled) {
ensureStartupMovieDefault(File(steam, "config/config.vdf"))
}
}
}
Loaded 100 of 372 files, more files were not shown because too many files have changed in this diff. Show more