mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-10-15 03:37:09 -04:00
MaxsTechReview
fb8361b197
* perf: fast path resolves links, /proc and glue as proot does, and answers uname, isatty and passwd
The fast path handed back to proot whatever crossed a symlink, /proc or a directory proot
only shows as glue, which is most of what Steam and Wine ask. It now walks a path the way
proot's canonicalize() does, so those calls stay in-process:
- links are followed in guest terms and their text detranslated as detranslate_path() does,
including readlink into a buffer the kernel cut short and /proc's exe, cwd and fd links;
- a binding proot dropped at start is dropped here too, glue above a binding is ENOENT, and a
last component proot cannot stat is left to the kernel;
- pipe and socket dirfds are ENOTDIR, rmdir("dir/") removes dir, and "." and ".." go to proot;
- a blocking accept() with a connection already waiting no longer goes to proot.
Also answered: uname() and gethostname() (proot's utsname never changes), isatty() (glibc
2.43 asks with TCGETS2, which proot rewrites) and getpwnam/getpwuid when nsswitch.conf
sends passwd to files first and the entry is there (Xwayland asks for every client).
equiv.py, equiv2.py and equiv3.py match line for line with and without the fast path
(222, 507 and 924 lines). Per call on the RedMagic: realpath 240 us -> 1.0 us,
realpath /proc/self/fd/N 408 -> 9.9, getpwuid 41.5 -> 4.6, uname 28 -> 0.006.
* perf: LD_AUDIT module skips library lookups proot would fail
ld.so looks each library up along the game's search path, and Proton puts the game's
directory, Steam's runtime and video directories and its own library directories ahead of
/usr/lib: 72% of ld.so's opens in an Among Us launch failed, each a round trip to proot.
libblaudit.so answers la_objsearch() for a candidate proot would certainly fail with ENOENT
(its directory walked as canonicalize() walks it, the name or a directory on the way missing).
It has no libc and maps the fast path's trampoline itself. Each lookup's first candidate is
still opened, since a skipped one fails with ld.so's errno as it stands, and the default
directories are never skipped, so dlerror() reads the same. droiddeck-proton and the extra
wrapper add it to LD_AUDIT when the fast path is on.
* perf: droiddeck-proton checks a tool's components with one sha256sum
Each program the launcher starts pays a dozen traced library lookups under the game's search
path before it does anything. One sha256sum over every listed file replaces one per file;
the check itself is unchanged.
* perf: build a locale archive for C.UTF-8
Every program loads the locale LANG names before main(). glibc ships C.UTF-8 as a directory
it looks for once per category under two spellings, before reading gconv's module lists:
about 30 opens, each a stop in proot, in every process of the session. droiddeck-session
builds /usr/lib/locale/locale-archive (and gconv's cache) once per glibc package, so that is
one open.
localedef names a finished archive by hard link, which Android refuses; with
DROIDDECK_LINK_RENAME=1 libblsession turns a refused link into a rename that will not replace
an existing name.
* perf: library beat starts its helpers only when what they read changed
Every 15 s the session ran droiddeck-seed-redists (some forty programs under proot) and, when
a library directory had changed, steam-compatibility: a burst of thousands of proot stops each
quarter minute, game loads included. The seeder now runs when a library changed or a prefix
(or Proton's template for them) is unmarked, the compat pass when a manifest, depot, tool or
config.vdf changed, both checked from bash without starting anything, and each every five
minutes regardless. The seeder reads its stamp and the time without a subshell each.
* perf: fakeinput opens pad.log on its first line
Every process the library is loaded in opened the session's pad.log at start, a stop in proot
on shared storage, and most never write to it. The file is now opened when there is a line to
write.
* docs: the fast path as it ships, the audit module and measured launch times
* fix: a game the app launches after an idle spell comes to the front
The Steam client brings a game it launched forward only while it counts someone as present,
which it reads from gamescope's input counter. A game the app launched after four or more
minutes without a touch started, mapped its window, and stayed behind the library page until
the next input (device: Palworld, Stardew Valley, Fallout: New Vegas, Mortal Kombat X; the same
on main). The session now watches a requested launch, and when the game's window is focusable
but the interface keeps focus for two polls, counts one input on gamescope's counter, as a key
would. Nothing changes for a launch the client already brings forward.
719 lines
35 KiB
YAML
719 lines
35 KiB
YAML
name: Build APK
|
|
run-name: ${{ github.event_name == 'pull_request' && format('building pr {0} on {1}', github.event.pull_request.number, github.event.pull_request.head.sha) || format('building {0} on {1}', github.ref_name, github.sha) }}
|
|
|
|
on:
|
|
push:
|
|
branches: [ main ]
|
|
paths-ignore:
|
|
- '**/*.md'
|
|
- 'LICENSE'
|
|
- '.gitignore'
|
|
pull_request:
|
|
branches: [ main ]
|
|
types: [ opened, synchronize, reopened, ready_for_review ]
|
|
paths-ignore:
|
|
- '**/*.md'
|
|
- 'LICENSE'
|
|
- '.gitignore'
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
contents: read
|
|
pull-requests: write
|
|
|
|
concurrency:
|
|
# Replace older builds only for the same PR. Each push/dispatch gets its own group so one main
|
|
# build cannot cancel another before it signs and publishes.
|
|
group: build-${{ github.event.pull_request.number || github.run_id }}
|
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
|
|
|
env:
|
|
PRELOAD_CACHE_VERSION: v3
|
|
|
|
jobs:
|
|
build:
|
|
name: Build APK
|
|
if: ${{ github.event_name != 'pull_request' || !github.event.pull_request.draft }}
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 45
|
|
outputs:
|
|
variants: ${{ steps.variants.outputs.list }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
|
|
fetch-depth: 0
|
|
|
|
- name: The packages a signed build ships as
|
|
id: variants
|
|
run: echo "list=$(awk '!/^#/ && NF { print $1 }' tools/release/variants.txt | jq -R . | jq -sc .)" >> "$GITHUB_OUTPUT"
|
|
|
|
- name: No private key in the tree
|
|
run: |
|
|
set -euo pipefail
|
|
keys=$(git ls-files | grep -iE '\.(jks|keystore|p12|pfx|pk8|pem|key)$' | grep -vx 'keystore/testkey.p12' || true)
|
|
if [ -n "$keys" ]; then
|
|
echo "::error::a private key is committed - remove it from the history and make a new one: $keys"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Version bumps are isolated
|
|
if: github.event_name == 'pull_request'
|
|
run: |
|
|
set -euo pipefail
|
|
git fetch --no-tags origin main
|
|
base=$(git merge-base origin/main HEAD)
|
|
diff=$(git diff -U0 "$base..HEAD" -- app/build.gradle)
|
|
grep -Eq '^[+-][[:space:]]*version(Code|Name)[[:space:]]' <<<"$diff" || exit 0
|
|
other_files=$(git diff --name-only "$base..HEAD" | grep -v '^app/build.gradle$' || true)
|
|
other_gradle=$(grep -E '^[+-]' <<<"$diff" | grep -Ev '^(---|\+\+\+)' | grep -Ev '^[+-][[:space:]]*version(Code|Name)[[:space:]]' || true)
|
|
if [[ -n "$other_files" || -n "$other_gradle" ]]; then
|
|
echo "::error::versionCode/versionName changes must be isolated in their own PR"
|
|
echo "A version PR may only change the versionCode/versionName lines in app/build.gradle." >> "$GITHUB_STEP_SUMMARY"
|
|
exit 1
|
|
fi
|
|
|
|
- name: Merge onto main, or say what conflicts
|
|
if: github.event_name == 'pull_request'
|
|
run: |
|
|
set -uo pipefail
|
|
git config user.name "ci"
|
|
git config user.email "[email protected]"
|
|
git fetch --no-tags origin main
|
|
BASE=$(git rev-parse --short origin/main)
|
|
HEAD=$(git rev-parse --short HEAD)
|
|
if git merge --no-ff --no-edit origin/main >/tmp/merge.out 2>&1; then
|
|
{
|
|
echo "### Mergeable"
|
|
echo
|
|
echo "\`$HEAD\` merges cleanly onto main (\`$BASE\`). What follows was built from the merged result."
|
|
} > /tmp/pr-merge.md
|
|
cat /tmp/pr-merge.md >> "$GITHUB_STEP_SUMMARY"
|
|
exit 0
|
|
fi
|
|
FILES=$(git diff --name-only --diff-filter=U)
|
|
if [ -z "$FILES" ]; then
|
|
{
|
|
echo "### ERROR: Could not merge onto main (\`$BASE\`)"
|
|
echo
|
|
echo '```'
|
|
cat /tmp/merge.out
|
|
echo '```'
|
|
} > /tmp/pr-merge.md
|
|
cat /tmp/pr-merge.md >> "$GITHUB_STEP_SUMMARY"
|
|
echo "::error::git could not merge main: $(head -3 /tmp/merge.out | tr '\n' ' ')"
|
|
exit 1
|
|
fi
|
|
{
|
|
echo "### ERROR: Not mergeable - conflicts with main (\`$BASE\`)"
|
|
echo
|
|
echo "Files that conflict:"
|
|
echo
|
|
for f in $FILES; do echo "- \`$f\`"; done
|
|
echo
|
|
echo "Each conflicting hunk, as git leaves it: \`<<<<<<< HEAD\` is this pull request, \`>>>>>>> origin/main\` is main."
|
|
echo
|
|
for f in $FILES; do
|
|
echo "<details><summary><code>$f</code></summary>"
|
|
echo
|
|
echo '```'
|
|
awk '/^<<<<<<< /{p=1} p{print} /^>>>>>>> /{p=0}' "$f" | head -300
|
|
echo '```'
|
|
echo "</details>"
|
|
echo
|
|
done
|
|
echo "Not built: rebase or merge main into the branch and push again."
|
|
} > /tmp/pr-merge.md
|
|
cat /tmp/pr-merge.md >> "$GITHUB_STEP_SUMMARY"
|
|
git merge --abort || true
|
|
echo "::error::the pull request conflicts with main: $(echo $FILES | tr '\n' ' ')"
|
|
exit 1
|
|
|
|
- uses: actions/setup-java@v4
|
|
with:
|
|
distribution: temurin
|
|
java-version: '17'
|
|
|
|
- uses: gradle/actions/setup-gradle@v4
|
|
with:
|
|
cache-cleanup: on-success
|
|
|
|
- name: Resolve the runner's NDK
|
|
run: |
|
|
NDK="${ANDROID_NDK_LATEST_HOME:-${ANDROID_NDK_HOME:-$ANDROID_NDK_ROOT}}"
|
|
VERSION=$(sed -n 's/^Pkg.Revision *= *//p' "$NDK/source.properties")
|
|
echo "NDK_VERSION=$VERSION" >> "$GITHUB_ENV"
|
|
|
|
- name: Set the APK build label
|
|
id: build-label
|
|
env:
|
|
PR_NUMBER: ${{ github.event.pull_request.number }}
|
|
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
|
BUILD_REF: ${{ github.ref_name }}
|
|
BUILD_SHA: ${{ github.sha }}
|
|
run: |
|
|
if [[ -n "$(git status --porcelain)" ]]; then
|
|
tree_state=dirty
|
|
else
|
|
tree_state=clean
|
|
fi
|
|
echo "DROIDDECK_BUILD_TREE_STATE=$tree_state" >> "$GITHUB_ENV"
|
|
if [[ -n "$PR_NUMBER" ]]; then
|
|
label="pr${PR_NUMBER} at ${PR_HEAD_SHA:0:7}"
|
|
else
|
|
label="${BUILD_REF} at ${BUILD_SHA:0:7}"
|
|
fi
|
|
echo "label=$label" >> "$GITHUB_OUTPUT"
|
|
echo "DROIDDECK_SOURCE_COMMIT=${PR_HEAD_SHA:-$BUILD_SHA}" >> "$GITHUB_ENV"
|
|
echo "DROIDDECK_PR_NUMBER=${PR_NUMBER:-0}" >> "$GITHUB_ENV"
|
|
echo "Build: \`$label\`" >> "$GITHUB_STEP_SUMMARY"
|
|
|
|
- name: Cache the preloads
|
|
id: preloads
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: app/src/main/assets/linuxfs
|
|
key: preloads-${{ env.PRELOAD_CACHE_VERSION }}-${{ hashFiles('app/src/main/cpp/fakeinput_steam.cpp', 'tools/linuxfs/preload/*.c', 'tools/linuxfs/build-x86-preloads.sh', 'tools/linuxfs/fex/*', 'tools/linuxfs/clipboard/*.c', 'tools/linuxfs/ssbs/*.c', 'tools/proot/fastpath/*.c', 'tools/linuxfs/overlay/usr/local/bin/*', 'tools/linuxfs/overlay/usr/bin/**', 'tools/linuxfs/desktop/*', 'tools/gamescope/release.env', 'tools/wlroots/release.env', 'tools/linuxfs/uruntime.env', 'tools/linuxfs/licenses/*', 'tools/mangoapp/*', 'tools/msitools/**') }}
|
|
|
|
- name: Build libfakeinput.so and libblsession.so (glibc aarch64)
|
|
if: steps.preloads.outputs.cache-hit != 'true'
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
run: |
|
|
set -euo pipefail
|
|
sudo apt-get update -qq
|
|
sudo apt-get install -y -qq gcc-aarch64-linux-gnu g++-aarch64-linux-gnu >/dev/null
|
|
D=app/src/main/assets/linuxfs
|
|
mkdir -p "$D"
|
|
aarch64-linux-gnu-g++ -shared -fPIC -O2 -Wall -Wno-attributes -Wno-nonnull-compare \
|
|
-pthread -std=c++17 -static-libstdc++ -static-libgcc -Wl,--exclude-libs,ALL \
|
|
-o "$D/libfakeinput.so" app/src/main/cpp/fakeinput_steam.cpp -ldl
|
|
aarch64-linux-gnu-strip --strip-unneeded "$D/libfakeinput.so"
|
|
aarch64-linux-gnu-gcc -shared -fPIC -O2 -Wall -pthread \
|
|
-o "$D/libblsession.so" tools/linuxfs/preload/*.c -ldl
|
|
aarch64-linux-gnu-strip --strip-unneeded "$D/libblsession.so"
|
|
aarch64-linux-gnu-gcc -shared -fPIC -O2 -Wall -pthread \
|
|
-o "$D/libblfastpath.so" tools/proot/fastpath/fastpath.c -ldl
|
|
aarch64-linux-gnu-strip --strip-unneeded "$D/libblfastpath.so"
|
|
aarch64-linux-gnu-gcc -shared -fPIC -O2 -Wall -nostdlib -ffreestanding -fno-stack-protector \
|
|
-fno-tree-loop-distribute-patterns -Wl,-z,defs -o "$D/libblaudit.so" tools/proot/fastpath/audit.c
|
|
aarch64-linux-gnu-strip --strip-unneeded "$D/libblaudit.so"
|
|
# Force-SSBS adapter, preloaded into a game's Wine only when the game asks for it.
|
|
aarch64-linux-gnu-gcc -shared -fPIC -O2 -Wall -pthread \
|
|
-o "$D/libssbs.so" tools/linuxfs/ssbs/ssbs_adapter.c -ldl
|
|
aarch64-linux-gnu-strip --strip-unneeded "$D/libssbs.so"
|
|
mkdir -p "$D/usr/local/bin"
|
|
aarch64-linux-gnu-gcc -O2 -Wall -Wextra -o "$D/usr/local/bin/droiddeck-clipboard" tools/linuxfs/clipboard/clipboard.c -ldl
|
|
aarch64-linux-gnu-strip --strip-unneeded "$D/usr/local/bin/droiddeck-clipboard"
|
|
for script in tools/linuxfs/overlay/usr/local/bin/droiddeck-* tools/linuxfs/overlay/usr/local/bin/steam-compatibility; do
|
|
install -Dm644 "$script" "$D/usr/local/bin/$(basename "$script")"
|
|
done
|
|
for f in tools/linuxfs/overlay/usr/bin/* tools/linuxfs/overlay/usr/bin/steamos-polkit-helpers/*; do
|
|
[ -f "$f" ] && install -Dm644 "$f" "$D/${f#tools/linuxfs/overlay/}"
|
|
done
|
|
if [ -f tools/gamescope/release.env ]; then
|
|
. tools/gamescope/release.env
|
|
gh release download "${GAMESCOPE_TAG}" -R "${GAMESCOPE_REPO:-${GITHUB_REPOSITORY}}" -p gamescope.tzst -O gamescope.tzst
|
|
echo "${GAMESCOPE_SHA256} gamescope.tzst" | sha256sum -c -
|
|
tar --use-compress-program=unzstd -xf gamescope.tzst -C "$D"
|
|
test -f "$D/usr/local/bin/gamescope"
|
|
rm gamescope.tzst
|
|
fi
|
|
. tools/linuxfs/uruntime.env
|
|
curl -fsSL --retry 3 -o uruntime "https://github.com/VHSgunzo/uruntime/releases/download/${URUNTIME_VERSION}/${URUNTIME_ASSET}"
|
|
echo "${URUNTIME_SHA256} uruntime" | sha256sum -c -
|
|
install -Dm644 uruntime "$D/usr/local/lib/droiddeck/uruntime"
|
|
install -Dm644 tools/linuxfs/licenses/uruntime-LICENSE "$D/usr/local/share/licenses/uruntime/LICENSE"
|
|
rm uruntime
|
|
M="$D/usr/local/lib/mangoapp"
|
|
mkdir -p "$M" mango-pkgs
|
|
grep -v '^#' tools/mangoapp/packages.txt | while read -r sum url; do
|
|
[ -n "$url" ] || continue
|
|
curl -fsSL --retry 3 -o mango-pkgs/pkg.tar.zst "$url"
|
|
echo "$sum mango-pkgs/pkg.tar.zst" | sha256sum -c -
|
|
tar --use-compress-program=unzstd -xf mango-pkgs/pkg.tar.zst -C mango-pkgs
|
|
rm mango-pkgs/pkg.tar.zst
|
|
done
|
|
install -m644 mango-pkgs/usr/bin/mangoapp "$M/mangoapp"
|
|
for so in libfmt.so.10 libspdlog.so.1.13 libglfw.so.3 libtraceevent.so.1; do
|
|
cp -L "mango-pkgs/usr/lib/$so" "$M/$so"
|
|
done
|
|
# Ours, not the package's: GPU memory without tracefs (tools/mangoapp/libtracefs-shim.c).
|
|
aarch64-linux-gnu-gcc -shared -fPIC -O2 -Wall -Wl,-soname,libtracefs.so.1 \
|
|
-o "$M/libtracefs.so.1" tools/mangoapp/libtracefs-shim.c
|
|
aarch64-linux-gnu-strip --strip-unneeded "$M/libtracefs.so.1"
|
|
install -Dm644 tools/mangoapp/mangoapp "$D/usr/local/bin/mangoapp"
|
|
rm -rf mango-pkgs
|
|
ls -l "$M"
|
|
# The Windows components installer's tools (tools/msitools/packages.txt).
|
|
T="$D/usr/local/lib/droiddeck-msitools"
|
|
mkdir -p "$T" msi-pkgs
|
|
grep -v '^#' tools/msitools/packages.txt | while read -r sum url; do
|
|
[ -n "$url" ] || continue
|
|
curl -fsSL --retry 3 -o msi-pkgs/pkg.tar.xz "$url"
|
|
echo "$sum msi-pkgs/pkg.tar.xz" | sha256sum -c -
|
|
tar -xJf msi-pkgs/pkg.tar.xz -C msi-pkgs
|
|
rm msi-pkgs/pkg.tar.xz
|
|
done
|
|
install -m644 msi-pkgs/usr/bin/msiinfo "$T/msiinfo"
|
|
install -m644 msi-pkgs/usr/bin/cabextract "$T/cabextract"
|
|
# 7-Zip (the full build with its format plugin) opens installer .exe wrappers.
|
|
install -m644 msi-pkgs/usr/lib/7zip/7z "$T/7z"
|
|
install -m644 msi-pkgs/usr/lib/7zip/7z.so "$T/7z.so"
|
|
for so in libmsi-1.0.so.0 libgsf-1.so.114 libgcab-1.0.so.0; do
|
|
cp -L "msi-pkgs/usr/lib/$so" "$T/$so"
|
|
done
|
|
for f in NOTICE GPL-2 GPL-3 LGPL-2.1 7zip-License 7zip-unRarLicense; do
|
|
install -Dm644 "tools/msitools/licenses/$f" "$D/usr/local/share/licenses/droiddeck-msitools/$f"
|
|
done
|
|
rm -rf msi-pkgs
|
|
ls -l "$T"
|
|
install -Dm644 tools/linuxfs/desktop/droiddeck-desktop "$D/usr/local/bin/droiddeck-desktop"
|
|
install -Dm644 tools/linuxfs/desktop/droiddeck-gpu "$D/usr/local/bin/droiddeck-gpu"
|
|
install -Dm644 tools/linuxfs/desktop/droiddeck-desktop-gpu "$D/usr/local/bin/droiddeck-desktop-gpu"
|
|
install -Dm644 tools/linuxfs/desktop/kwin_wayland_wrapper "$D/usr/local/bin/kwin_wayland_wrapper"
|
|
install -Dm644 tools/linuxfs/desktop/firefox-droiddeck.js "$D/usr/lib/firefox/defaults/pref/droiddeck.js"
|
|
ls -lR "$D"
|
|
|
|
- name: Build the x86 preloads FEX guests load (glibc 2.31)
|
|
if: steps.preloads.outputs.cache-hit != 'true'
|
|
run: |
|
|
set -euo pipefail
|
|
docker run --rm -v "$PWD:/src" -w /src public.ecr.aws/docker/library/debian:bullseye bash -c '
|
|
set -euo pipefail
|
|
printf "deb http://archive.debian.org/debian bullseye main\ndeb http://archive.debian.org/debian-security bullseye-security main\n" > /etc/apt/sources.list
|
|
apt-get -o Acquire::Check-Valid-Until=false update -qq
|
|
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq --no-install-recommends gcc g++ gcc-multilib g++-multilib binutils >/dev/null
|
|
tools/linuxfs/build-x86-preloads.sh app/src/main/assets/linuxfs
|
|
'
|
|
sudo chown -R "$(id -u):$(id -g)" app/src/main/assets/linuxfs
|
|
ls -l app/src/main/assets/linuxfs/x86_64 app/src/main/assets/linuxfs/i386
|
|
|
|
- name: Bundle the droiddeck-esync packs
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ -f tools/droiddeck-esync/release.env ]; then
|
|
. tools/droiddeck-esync/release.env
|
|
gh release download "$SYNC_BUNDLE_TAG" -R "$SYNC_BUNDLE_REPO" -p "$SYNC_BUNDLE_ASSET" -O sync-bundle.tzst
|
|
echo "$SYNC_BUNDLE_SHA256 sync-bundle.tzst" | sha256sum -c -
|
|
mkdir -p app/src/main/assets/droiddeck-esync
|
|
tar --use-compress-program=unzstd -xf sync-bundle.tzst -C app/src/main/assets/droiddeck-esync
|
|
rm sync-bundle.tzst
|
|
gh release download "$SYNC_BUNDLE_TAG" -R "$SYNC_BUNDLE_REPO" -p index.json -O sync-index.json
|
|
revoked=$(python3 -c 'import json, sys; print(" ".join(p["id"] for p in json.load(open(sys.argv[1]))["packs"] if p.get("revoked") is True))' sync-index.json)
|
|
for id in $revoked; do
|
|
if [ -e "app/src/main/assets/droiddeck-esync/packs/$id.tzst" ]; then
|
|
echo "::warning::$SYNC_BUNDLE_ASSET carries revoked pack $id; it is left out of the APK (point tools/droiddeck-esync/release.env at a newer bundle)"
|
|
rm -f "app/src/main/assets/droiddeck-esync/packs/$id.tzst"
|
|
fi
|
|
done
|
|
rm sync-index.json
|
|
test -f app/src/main/assets/droiddeck-esync/index.json
|
|
test -f app/src/main/assets/droiddeck-esync/index.json.sig
|
|
ls -lR app/src/main/assets/droiddeck-esync
|
|
fi
|
|
|
|
- name: Cache PulseAudio 13.0 source
|
|
id: pa-src
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: pa13
|
|
key: pulseaudio-src-13.0
|
|
- name: Fetch PulseAudio 13.0 source
|
|
if: steps.pa-src.outputs.cache-hit != 'true'
|
|
run: |
|
|
set -euo pipefail
|
|
curl -fsSL -o pa.tar.gz https://github.com/pulseaudio/pulseaudio/archive/refs/tags/v13.0.tar.gz
|
|
mkdir -p pa13 && tar -xzf pa.tar.gz -C pa13 --strip-components=1 && rm pa.tar.gz
|
|
# DirectAudio - the game driver (one set per Wine audio interface), the Android-side helper and
|
|
# the Steam client's sinks - comes from a pinned release of The412Banner/directaudio, checksummed,
|
|
# exactly like gamescope; nothing compiled for audio is kept in this repo.
|
|
- name: Fetch DirectAudio (tools/directaudio/release.env)
|
|
run: tools/directaudio/fetch.sh "$PWD" "$PWD/sink-out"
|
|
- name: Build the AAudio sink into the audio bundle
|
|
run: |
|
|
set -euo pipefail
|
|
export NDK="${ANDROID_NDK_LATEST_HOME:-${ANDROID_NDK_HOME:-$ANDROID_NDK_ROOT}}"
|
|
tools/aaudio-sink/build.sh "$PWD/pa13" "$PWD/sink-out"
|
|
rm -rf pa-bundle && mkdir -p pa-bundle
|
|
tar --use-compress-program=unzstd -xf app/src/main/assets/pulseaudio.tzst -C pa-bundle
|
|
test ! -f pa-bundle/modules/arm64/module-aaudio-sink.so
|
|
install -m755 sink-out/module-aaudio-sink.so pa-bundle/modules/arm64/module-aaudio-sink.so
|
|
install -m755 sink-out/module-directaudio-native-sink.so pa-bundle/modules/arm64/module-directaudio-native-sink.so
|
|
(cd pa-bundle && tar --use-compress-program='zstd -19' -cf ../pulseaudio.tzst .)
|
|
mkdir -p app/build/prepared-assets
|
|
mv pulseaudio.tzst app/build/prepared-assets/pulseaudio.tzst
|
|
tar --use-compress-program=unzstd -tf app/build/prepared-assets/pulseaudio.tzst | grep -c '\.so$'
|
|
rm -rf pa-bundle sink-out
|
|
- name: Cache proot
|
|
id: proot
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: |
|
|
app/src/main/jniLibs/arm64-v8a/libproot.so
|
|
app/src/main/jniLibs/arm64-v8a/libproot-loader.so
|
|
key: proot-${{ env.NDK_VERSION }}-${{ hashFiles('tools/proot/**') }}
|
|
- name: Build proot
|
|
if: steps.proot.outputs.cache-hit != 'true'
|
|
run: |
|
|
set -euo pipefail
|
|
export NDK="${ANDROID_NDK_LATEST_HOME:-${ANDROID_NDK_HOME:-$ANDROID_NDK_ROOT}}"
|
|
tools/proot/build.sh app/src/main/jniLibs/arm64-v8a
|
|
|
|
- name: Check the preloads
|
|
run: |
|
|
set -euo pipefail
|
|
bash tools/linuxfs/check-preloads.sh app/src/main/assets/linuxfs
|
|
|
|
# The store download engines (app/src/main/rust/stores) are Rust, built by Gradle's
|
|
# buildRustStores task through cargo-ndk. rustup and cargo are on the runner image; the Android
|
|
# target and cargo-ndk are added here. Cache = cargo registry + cargo-ndk + the crate's target
|
|
# tree, keyed on the rustc version and Cargo.lock, so a source-only change rebuilds this crate
|
|
# on top of the cached dependency objects, and a lockfile change still starts from the
|
|
# previous objects (restore-keys).
|
|
- name: Rust toolchain for the store engines (aarch64-linux-android)
|
|
id: rust
|
|
run: |
|
|
set -euo pipefail
|
|
rustup target add aarch64-linux-android
|
|
echo "rustc=$(rustc -V | awk '{print $2}')" >> "$GITHUB_OUTPUT"
|
|
rustc -V; cargo -V
|
|
|
|
- name: Cache cargo registry and the store engines' target
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: |
|
|
~/.cargo/registry/index
|
|
~/.cargo/registry/cache
|
|
~/.cargo/git/db
|
|
~/.cargo/bin/cargo-ndk
|
|
app/src/main/rust/stores/target
|
|
key: stores-${{ steps.rust.outputs.rustc }}-${{ hashFiles('app/src/main/rust/stores/Cargo.lock') }}
|
|
restore-keys: |
|
|
stores-${{ steps.rust.outputs.rustc }}-
|
|
|
|
- name: cargo-ndk
|
|
run: |
|
|
set -euo pipefail
|
|
command -v cargo-ndk >/dev/null || cargo install cargo-ndk --locked
|
|
cargo ndk --version
|
|
|
|
- name: Cache the CMake build tree
|
|
uses: actions/cache@v4
|
|
with:
|
|
path: |
|
|
app/.cxx
|
|
app/build/intermediates/cxx
|
|
key: cxx-${{ env.NDK_VERSION }}-${{ hashFiles('app/src/main/cpp/**') }}
|
|
restore-keys: |
|
|
cxx-${{ env.NDK_VERSION }}-
|
|
|
|
- name: Build
|
|
id: build
|
|
env:
|
|
DROIDDECK_BUILD_LABEL: ${{ steps.build-label.outputs.label }}
|
|
SGDB_API_KEY: ${{ secrets.SGDB_API_KEY }}
|
|
run: |
|
|
set -o pipefail
|
|
python3 tools/release/runtime_inputs.py record
|
|
./gradlew assembleRelease --console=plain -PndkVersion="$NDK_VERSION" 2>&1 | tee /tmp/build.log
|
|
python3 tools/release/check_session_assets.py app/build/outputs/apk/release/app-release.apk
|
|
python3 tools/release/runtime_inputs.py apk app/build/outputs/apk/release/app-release.apk
|
|
|
|
- name: Check the store engines' library
|
|
# The exports Kotlin binds must be in the apk's libdroiddeckstores.so, or the stores fail on
|
|
# the device with an UnsatisfiedLinkError instead of here. The lockfile check is the
|
|
# reproducibility guard: cargo must not have had to re-resolve anything.
|
|
run: |
|
|
set -euo pipefail
|
|
NDK="${ANDROID_NDK_LATEST_HOME:-${ANDROID_NDK_HOME:-$ANDROID_NDK_ROOT}}"
|
|
python3 tools/release/check_android_libraries.py app/build/outputs/apk/release/app-release.apk \
|
|
--readelf "$NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-readelf"
|
|
git diff --stat -- app/src/main/rust/stores/Cargo.lock
|
|
git diff --exit-code -- app/src/main/rust/stores/Cargo.lock \
|
|
|| { echo "::error::Cargo.lock changed during the build; commit the regenerated lockfile (uploaded as stores-cargo-lock)"; exit 1; }
|
|
|
|
- name: Store engines host unit tests
|
|
# The crate's #[test] functions are pure host-side logic (manifest parsing, chunk plans,
|
|
# assembly from a cache, hashes), so a plain `cargo test` on the runner's own architecture
|
|
# runs them; the Android build above compiles none of them. Informational on this first
|
|
# run - the point is to learn whether they pass - and bounded so a hang can never hold
|
|
# the apk. Make it blocking once it is known green.
|
|
timeout-minutes: 10
|
|
continue-on-error: true
|
|
run: |
|
|
cd app/src/main/rust/stores
|
|
cargo test --locked 2>&1 | tail -60
|
|
|
|
- name: Upload the regenerated Cargo.lock
|
|
if: failure()
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: stores-cargo-lock
|
|
path: app/src/main/rust/stores/Cargo.lock
|
|
if-no-files-found: ignore
|
|
|
|
- name: Steam import and game environment tests
|
|
run: python3 -m unittest discover -s tools/tests -p 'test_*.py'
|
|
|
|
- name: Unit tests
|
|
run: ./gradlew testReleaseUnitTest --console=plain -PndkVersion="$NDK_VERSION"
|
|
|
|
- name: Sign with the testkey (v1 + v2 + v3)
|
|
run: |
|
|
set -euo pipefail
|
|
APK=$(find app/build/outputs/apk -name '*.apk' | head -1)
|
|
TOOLS=$(find "$ANDROID_HOME/build-tools" -maxdepth 1 -type d | sort -V | tail -1)
|
|
"$TOOLS/zipalign" -p -f 4 "$APK" "$APK.aligned"
|
|
"$TOOLS/apksigner" sign \
|
|
--ks keystore/testkey.p12 --ks-type PKCS12 --ks-pass pass:android \
|
|
--ks-key-alias testkey --key-pass pass:android \
|
|
--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true \
|
|
--out "$APK" "$APK.aligned"
|
|
rm -f "$APK.aligned" "$APK.idsig"
|
|
|
|
- name: Check the signature
|
|
run: |
|
|
set -euo pipefail
|
|
APK=$(find app/build/outputs/apk -name '*.apk' | head -1)
|
|
APKSIGNER=$(find "$ANDROID_HOME/build-tools" -name apksigner | sort -V | tail -1)
|
|
"$APKSIGNER" verify --min-sdk-version 21 --verbose --print-certs "$APK" | tee /tmp/sig.txt
|
|
unzip -l "$APK" | grep -qE 'META-INF/.*\.(SF|RSA|DSA)$' \
|
|
|| { echo "::error::apk carries no JAR signature files"; exit 1; }
|
|
for scheme in "v1 scheme (JAR signing): true" "v2 scheme (APK Signature Scheme v2): true" \
|
|
"v3 scheme (APK Signature Scheme v3): true"; do
|
|
grep -qF "$scheme" /tmp/sig.txt || { echo "::error::apk is not signed with $scheme"; exit 1; }
|
|
done
|
|
grep -qi "CN=Android, OU=Android, O=Android" /tmp/sig.txt \
|
|
|| { echo "::error::apk is not signed with the AOSP testkey"; exit 1; }
|
|
|
|
- name: Report on the pull request
|
|
if: always() && github.event_name == 'pull_request'
|
|
continue-on-error: true
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
PR: ${{ github.event.pull_request.number }}
|
|
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
|
|
BUILD_OUTCOME: ${{ steps.build.outcome }}
|
|
SAME_REPO: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
|
|
run: |
|
|
set -uo pipefail
|
|
MARK="<!-- droiddeck-pr-build -->"
|
|
{
|
|
echo "$MARK"
|
|
echo "## Build of this pull request"
|
|
echo
|
|
[ -f /tmp/pr-merge.md ] && cat /tmp/pr-merge.md
|
|
echo
|
|
case "$BUILD_OUTCOME" in
|
|
success)
|
|
echo "### Build passed"
|
|
echo
|
|
echo "The \`droiddeck-apk\` artifact is [on this run]($RUN_URL), signed with the public test key."
|
|
echo
|
|
if [ "$SAME_REPO" = true ]; then
|
|
echo "A release-signed tester build that installs over Stable or Preview is being published now and will appear in DroidDeck's **Test builds** list. Label the PR \`no-test-build\` to skip that."
|
|
else
|
|
echo "**Maintainer:** add the \`publish-test\` label to publish a release-signed tester build that installs over Stable or Preview and appears in DroidDeck's **Test builds** list. The label is removed automatically after publishing."
|
|
fi
|
|
echo
|
|
echo "Fallback: [open **PR Release Publish**]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/workflows/pr-release-publish.yml) on \`main\` and enter PR \`$PR\`."
|
|
;;
|
|
failure)
|
|
echo "### ERROR: Build failed"
|
|
echo
|
|
echo "The compiler's errors ([full log]($RUN_URL)):"
|
|
echo
|
|
echo '```'
|
|
{ grep -E '^e: |^w: .*error|error:|CMake Error|fatal error|ninja: error|FAILED|What went wrong|Execution failed for task|Unresolved reference|Type mismatch|Unresolved' /tmp/build.log 2>/dev/null \
|
|
| sed -e 's#file:///home/runner/work/[^/]*/[^/]*/##' -e 's#/home/runner/work/[^/]*/[^/]*/##' \
|
|
| grep -v '^w: ' | awk '!seen[$0]++' | head -80; } || true
|
|
echo '```'
|
|
echo
|
|
echo "<details><summary>What went wrong (Gradle)</summary>"
|
|
echo
|
|
echo '```'
|
|
awk '/^\* What went wrong/{p=1} /^\* Try:/{p=0} p{print}' /tmp/build.log 2>/dev/null | head -60
|
|
echo '```'
|
|
echo "</details>"
|
|
;;
|
|
skipped|cancelled)
|
|
echo "### Not built"
|
|
echo
|
|
echo "The build did not run (see above, or [the run]($RUN_URL))."
|
|
;;
|
|
esac
|
|
echo
|
|
} > /tmp/pr-report.md
|
|
cat /tmp/pr-report.md >> "$GITHUB_STEP_SUMMARY"
|
|
ID=$(gh api "repos/${GITHUB_REPOSITORY}/issues/${PR}/comments" --paginate --jq ".[] | select(.body | startswith(\"$MARK\")) | .id" | head -1 || true)
|
|
if [ -n "$ID" ]; then
|
|
gh api -X PATCH "repos/${GITHUB_REPOSITORY}/issues/comments/${ID}" -F body=@/tmp/pr-report.md >/dev/null
|
|
else
|
|
gh api -X POST "repos/${GITHUB_REPOSITORY}/issues/${PR}/comments" -F body=@/tmp/pr-report.md >/dev/null
|
|
fi
|
|
|
|
- uses: actions/upload-artifact@v4
|
|
with:
|
|
name: droiddeck-apk
|
|
path: app/build/outputs/apk/release/*.apk
|
|
if-no-files-found: error
|
|
retention-days: 30
|
|
compression-level: 0
|
|
|
|
sign:
|
|
needs: build
|
|
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
|
|
permissions:
|
|
contents: read
|
|
actions: read
|
|
uses: ./.github/workflows/sign.yml
|
|
with:
|
|
environment: signing-main
|
|
variants: ${{ needs.build.outputs.variants }}
|
|
secrets: inherit
|
|
|
|
# Each signed main build as a prerelease of the public Droid-Deck/DroidDeck-CI (main-<sha>, the
|
|
# last KEEP kept), where anyone can download it without a GitHub account, then a post in the
|
|
# Discord's builds channel linking to it. The release is made with a token of the org's
|
|
# "DroidDeck CI Publisher" app (installed on DroidDeck-CI only, Contents: write): its ID in the
|
|
# DROIDDECK_CI_APP_ID variable, its private key in the DROIDDECK_CI_APP_KEY secret.
|
|
# DISCORD_BUILDS_WEBHOOK is the channel's webhook URL.
|
|
announce:
|
|
needs: [build, sign]
|
|
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
|
|
runs-on: ubuntu-24.04
|
|
timeout-minutes: 10
|
|
permissions:
|
|
contents: read
|
|
pull-requests: read
|
|
env:
|
|
CI_REPO: Droid-Deck/DroidDeck-CI
|
|
KEEP: 10
|
|
steps:
|
|
- uses: actions/download-artifact@v4
|
|
with:
|
|
pattern: droiddeck-signed-*
|
|
merge-multiple: true
|
|
path: signed
|
|
|
|
- uses: actions/create-github-app-token@v2
|
|
id: ci-token
|
|
with:
|
|
app-id: ${{ vars.DROIDDECK_CI_APP_ID }}
|
|
private-key: ${{ secrets.DROIDDECK_CI_APP_KEY }}
|
|
owner: Droid-Deck
|
|
repositories: DroidDeck-CI
|
|
|
|
# What this build is, for the top of the release notes: the merged PR's title and the first
|
|
# paragraph of its description, or the commit's subject for a push made without a PR.
|
|
- name: Describe the change
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
run: |
|
|
set -euo pipefail
|
|
pr=$(gh api "repos/$GITHUB_REPOSITORY/commits/$GITHUB_SHA/pulls" \
|
|
--jq '[.[] | select(.merged_at != null and .base.ref == "main")] | first // empty')
|
|
if [ -n "$pr" ]; then
|
|
number=$(jq -r .number <<<"$pr")
|
|
title=$(jq -r .title <<<"$pr")
|
|
merged=$(jq -r '.merged_at[:10]' <<<"$pr")
|
|
summary=$(jq -r '.body // ""' <<<"$pr" | tr -d '\r' | { grep -v '^🤖' || true; } |
|
|
awk '!p && /^#/ { next } NF { p = 1; print; next } p { exit }' | head -n 6 | cut -c1-400)
|
|
{
|
|
echo "**Merged to main** - [PR #$number]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/pull/$number): $title (merged $merged)"
|
|
if [ -n "$summary" ]; then
|
|
echo
|
|
echo "$summary"
|
|
fi
|
|
} > notes-head.md
|
|
else
|
|
subject=$(gh api "repos/$GITHUB_REPOSITORY/commits/$GITHUB_SHA" --jq '.commit.message | split("\n")[0]')
|
|
echo "**Pushed to main** without a PR - $subject" > notes-head.md
|
|
fi
|
|
cat notes-head.md
|
|
|
|
- name: Publish to DroidDeck-CI
|
|
id: publish
|
|
env:
|
|
GH_TOKEN: ${{ steps.ci-token.outputs.token }}
|
|
run: |
|
|
set -euo pipefail
|
|
tag=main-${GITHUB_SHA::7}
|
|
cat signed/SHA256SUMS-*.txt > signed/SHA256SUMS.txt
|
|
rm signed/SHA256SUMS-*.txt
|
|
notes="$(cat notes-head.md)
|
|
|
|
Signed build of $GITHUB_SERVER_URL/$GITHUB_REPOSITORY/commit/$GITHUB_SHA - in main, not yet in a DroidDeck release; for testing.
|
|
Commit: $GITHUB_SHA
|
|
VersionCode: $(sed -n 's/^ *versionCode \([0-9][0-9]*\)$/\1/p' app/build.gradle)
|
|
Each apk installs over a DroidDeck release and keeps its data. \`standard\` is the normal app; the others are the same app under another package name, installed beside it.
|
|
|
|
SHA-256:
|
|
\`\`\`
|
|
$(cat signed/SHA256SUMS.txt)
|
|
\`\`\`"
|
|
gh release delete "$tag" -R "$CI_REPO" --yes --cleanup-tag 2>/dev/null || true
|
|
gh release create "$tag" -R "$CI_REPO" --target main --prerelease \
|
|
--title "main ${GITHUB_SHA::7}" --notes "$notes" signed/*
|
|
# Keep the newest main builds by publication time. GitHub App releases can share createdAt.
|
|
gh release list -R "$CI_REPO" -L 200 --json tagName,publishedAt \
|
|
--jq '[.[] | select(.tagName | startswith("main-"))] | sort_by(.publishedAt) | reverse | .['"$KEEP"':][] | .tagName' |
|
|
while IFS= read -r old; do
|
|
# Never remove this run's release, even if the API returns an unexpected ordering.
|
|
if [ "$old" = "$tag" ]; then
|
|
continue
|
|
fi
|
|
# Another main publish may have removed the same old release concurrently.
|
|
if ! gh release delete "$old" -R "$CI_REPO" --yes --cleanup-tag 2>/dev/null; then
|
|
if gh release view "$old" -R "$CI_REPO" >/dev/null 2>&1; then
|
|
echo "::error::Could not delete old release $old"
|
|
exit 1
|
|
fi
|
|
fi
|
|
done
|
|
published=$(gh release view "$tag" -R "$CI_REPO" --json tagName,assets)
|
|
jq -e --arg tag "$tag" '.tagName == $tag and (.assets | length) > 0' <<< "$published" >/dev/null
|
|
url="$GITHUB_SERVER_URL/$CI_REPO/releases/tag/$tag"
|
|
apk=$(cd signed && ls DroidDeck-*.apk | awk 'length(a) == 0 || length($0) < length(a) { a = $0 } END { print a }')
|
|
echo "release=$url" >> "$GITHUB_OUTPUT"
|
|
echo "apk=$GITHUB_SERVER_URL/$CI_REPO/releases/download/$tag/$apk" >> "$GITHUB_OUTPUT"
|
|
echo "Published $url"
|
|
|
|
- name: Post to Discord
|
|
env:
|
|
WEBHOOK: ${{ secrets.DISCORD_BUILDS_WEBHOOK }}
|
|
GH_TOKEN: ${{ github.token }}
|
|
RELEASE: ${{ steps.publish.outputs.release }}
|
|
APK: ${{ steps.publish.outputs.apk }}
|
|
run: |
|
|
set -euo pipefail
|
|
if [ -z "$WEBHOOK" ]; then
|
|
echo "::notice::No DISCORD_BUILDS_WEBHOOK secret; nothing posted"
|
|
exit 0
|
|
fi
|
|
commit=$(gh api "repos/$GITHUB_REPOSITORY/commits/$GITHUB_SHA")
|
|
subject=$(jq -r '.commit.message | split("\n")[0]' <<<"$commit")
|
|
# A merge commit's subject is "Merge pull request #N from ..."; its body's first line is
|
|
# the PR's title, which says more.
|
|
title=$(jq -r '.commit.message | split("\n") | map(select(length > 0)) | .[1] // ""' <<<"$commit")
|
|
pr=$(sed -n 's/^Merge pull request #\([0-9]*\).*/\1/p' <<<"$subject")
|
|
if [ -n "$pr" ]; then
|
|
what="[#$pr]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/pull/$pr): ${title:-$subject}"
|
|
else
|
|
what="[\`${GITHUB_SHA::7}\`]($GITHUB_SERVER_URL/$GITHUB_REPOSITORY/commit/$GITHUB_SHA): $subject"
|
|
fi
|
|
get=$(printf 'Download: [standard APK](<%s>) · [all variants](<%s>)' "$APK" "$RELEASE")
|
|
content=$(printf '**New signed build** `main-%s`\n%s\n%s' "${GITHUB_SHA::7}" "$what" "$get")
|
|
jq -n --arg content "$content" '{content: $content, allowed_mentions: {parse: []}}' |
|
|
curl -fsS -H 'Content-Type: application/json' -d @- "$WEBHOOK" -o /dev/null
|
|
echo "Posted: $content"
|
|
|
|
catalog:
|
|
needs: announce
|
|
if: ${{ always() && github.event_name != 'pull_request' && github.ref == 'refs/heads/main' && needs.announce.result != 'skipped' }}
|
|
uses: ./.github/workflows/update-catalog.yml
|
|
secrets: inherit
|