Add native Crystal exploration port and reference tooling
Checks / web (push) Canceled after 0s

This commit is contained in:
Sébastien QUEROL committed 2026-09-23 09:23:29 +02:00
1 parent 2127f5f459
commit 8b80d05ca2
56 files changed
+6910 -1

No files matched your search

+44
View File
@@ -0,0 +1,44 @@
# Architecture
The default application is a native TypeScript game. It loads no CPU interpreter, Game Boy hardware model or WebAssembly emulator. The reference emulator has a separate entry point and build.
```mermaid
flowchart LR
ROM[User ROM] --> Gate[Exact revision validation]
Gate --> Assets[ROM reader and asset decoders]
Assets --> World[Native world rules]
Assets --> Scripts[Native event and text interpreters]
Scripts --> World
Input[Touch and keyboard] --> Runtime[Native runtime]
Runtime --> World
World --> View[Canvas world and DOM dialogue]
Session[Application session] --> Runtime
Session --> Saves[Native save storage]
Reference[Separate reference emulator] -. local comparisons only .-> World
```
| Module | Responsibility |
| ------------------------------- | ------------------------------------------------------------------------------------------- |
| `src/rom/validate.ts` | Exact cartridge identity, no networking |
| `src/game/rom` | Banked reads, LZ, maps/tilesets/sprites/palettes and pinned addresses |
| `src/game/world` | Position, movement, collision, warps, visibility, time selection and native save validation |
| `src/game/scripts` | The game's event/text formats, no GB instructions |
| `src/game/rendering` | Planar pixels, metatiles, camera and sprites |
| `src/game/runtime.ts` | Frame scheduling, input, interpolation and presentation updates |
| `src/input` | Abstract buttons, multiple sources and browser bindings |
| `src/app` | Runtime-independent load/play/pause/backup lifecycle |
| `src/storage` | Versioned envelope, IndexedDB transactions and ordered writes |
| `src/ui`, `src/main.ts` | UI composition and DOM dialogue, no ROM offsets |
| `src/platform` | Browser locks and future native platform adapters |
| `src/reference`, `src/emulator` | Original-game reference, excluded from the native build |
| `reference/public` | Licensed emulator files for the reference build only |
`GameRuntime` is a lifecycle interface, not an emulator contract. Native state is validated JSON: map, position, facing, name/avatar, in-game tile count, flags and sprite variables. It contains no graphics, music, dialogue, GB memory or ROM bytes. The backup envelope retains the historical `coreId` field; native backups use it for their runtime/schema ID.
Native and reference sessions use different IndexedDB databases and Web Locks. Imports reject incompatible schemas and invalid positions before changing the world. A failed import restores the prior state. Original cartridge saves and emulator snapshots cannot be used as native saves.
Scripts evaluate against a private draft. Unknown commands abort it. Dialogue commits only after the last page is acknowledged. A mid-dialogue save contains the last committed state, allowing the interaction to be replayed without partial rewards/flags. Automatic story scenes and callbacks are explicitly disabled in the exploration milestone.
Asset caches and the ROM reader remain in memory for lazy extraction. Neither ROMs nor extracted assets enter IndexedDB, the build or a backend. Vite blocks serving the root ROM and private reference checkout during development.
Introduce future modules with working features: `game/battle`, `game/party`, `game/inventory`, `game/audio`, then `features/link` and `features/walking`. Rules must not import UI/browser modules. Transports and step providers must not write directly into world state.
+60
View File
@@ -0,0 +1,60 @@
# Native port status and reference checks
## Reproducible reference
The private `.reference/pokecrystal` checkout is pinned to `7a7881d0d62e0ddbd82dcf10e7116807487ac651`. With RGBDS 1.0.3, its `crystal11` build is byte-identical to the supplied English US/Europe revision 1.1 ROM. `npm run check:reference` verifies the files, commit and every address in `src/game/rom/profile.ts` against generated symbols.
To recreate the private reference:
```sh
git clone https://github.com/pret/pokecrystal.git .reference/pokecrystal
git -C .reference/pokecrystal checkout 7a7881d0d62e0ddbd82dcf10e7116807487ac651
make -C .reference/pokecrystal crystal11 -j4
npm run check:reference
```
Install RGBDS 1.0.3 separately. No script downloads a game ROM. `CRYSTAL_ROM_PATH` optionally selects another local location; checks otherwise use `Pokemon - Crystal Version (UE) (V1.1) [C][!].gbc` at the root.
The reference checkout, rebuilt ROM, extracted graphics, saves and gameplay screenshots must stay local and ignored. The shipping profile contains addresses and format constants, not game content.
## Routine map
These paths refer to the pinned [PRET source](https://github.com/pret/pokecrystal/tree/7a7881d0d62e0ddbd82dcf10e7116807487ac651).
| Native module | PRET reference | Evidence and limits |
| -------------------------------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `game/rom/lz.ts` | `home/decompress.asm` | All LZ command types have unit vectors; six tilesets match original ASM execution byte-for-byte. |
| `game/rom/assets.ts` maps/events | `data/maps/maps.asm`, `data/maps/attributes.asm`, `macros/scripts/maps.asm` | Six-map dimensions, events, destinations, navigation and return warps checked with the actual ROM. Connections are decoded but traversal is disabled. |
| Tiles/palettes | `home/map.asm`, `engine/gfx/color.asm`, `engine/tilesets/mapgroup_roofs.asm` | Graphics match rebuilt reference bytes. Day/night palettes and roofs supported; terrain animation/dynamic tiles and other special palettes pending. |
| Sprites/rendering | `data/sprites/sprites.asm`, `data/sprites/facings.asm` | Chris/Kris and standard NPC graphics/facings decoded. Dynamic Pokémon icons and room decorations pending. |
| `game/world/world.ts` | `engine/overworld/player_movement.asm`, `data/collision/collision_permissions.asm` | Walking, walls, visible objects and ordinary local warps. Not complete frame-by-frame parity; forced movement, ledges and surfing are refused. |
| Event interpreter | `macros/scripts/events.asm`, `engine/overworld/scripting.asm`, `engine/events/std_scripts.asm` | Bounded calls/jumps/conditions, flags, initial sprite variables and basic text interactions. Unknown commands abort; no assembly fallback. |
| Text interpreter | `macros/scripts/text.asm`, `constants/charmap.asm` | Literal/far text, line/paragraph controls and player substitution. Dynamic RAM/ASM text is unsupported. |
Supported event opcodes are explicit in the interpreter switch. Audio commands and delays consume their operands, but sound sequencing and presentation timing are pending; those opcodes do not imply complete audiovisual behavior.
## Actual emulator comparison
`tests/reference-parity.spec.ts` loads the user's ROM into the reference emulator and places a small original test trampoline in emulated WRAM. It calls the ROM's `FarDecompress` routine at `$0b40`, then sets a return sentinel. Every output byte is compared with the native decoder for six supported map tilesets. This executes the original routine rather than comparing two copies of our code.
That establishes decompression parity for those inputs. It does not prove complete gameplay, rendering, battles, audio, timing or physical-device parity. Extend controlled comparisons as each subsystem is translated.
## Validation
```sh
npm test
npm run test:e2e
npm run check:reference
npm run check:vendor
npm run build
npm run check:release
npm run build:reference
```
User-ROM tests skip when the file is absent, including public CI. Synthetic tests still run. Comparing with rebuilt graphics also needs the private PRET build. Do not report skipped private checks as CI passes.
Native production is `dist/`; the optional reference output is `dist-reference/`. Release checks reject emulator dependencies, binary game files and embedded binary assets in the native output. Source review is still required when adding assets or data.
## Next slice
Expand automatic scenes/callbacks and native party/inventory models to implement the opening through starter selection. Add the first battle loop with Route 29 and the Mystery Egg quest. Keep this story milestone separate from exploration saves until initialization and migration are defined.
+88
View File
@@ -0,0 +1,88 @@
# Full native port plan
## Direction
Translate Crystal's Game Boy assembly systems into a native TypeScript game. The browser runs the translated engine; the user's original ROM supplies data and assets. The emulator is a separate development reference for checking behavior, never a dependency of the native release.
PRET's [disassembly](https://github.com/pret/pokecrystal) is pinned at `7a7881d0d62e0ddbd82dcf10e7116807487ac651`. Its `crystal11` build with RGBDS 1.0.3 was verified byte-for-byte against the supplied English US/Europe 1.1 ROM. Required SHA-1: `f2f52230b536214ef7c9924f483392993e226cfb`. Other revisions require separate verified profiles.
The emulator-backed prototype is now at `/reference.html`. The default application is the native port. The previous plan to progressively modify an emulated ROM has been superseded.
## Implemented: first native exploration milestone
- Bounds-checked ROM access, banked pointers and native LZ decompression.
- On-demand extraction of six maps: New Bark Town, both floors of the player's home, Elm's lab, the neighbor's house and Elm's house.
- Metatiles, collision quadrants, palettes, roofs, sprites and facing tables read directly from the user's ROM.
- Native canvas rendering, camera scrolling, walking, solid-tile/NPC collision, doors and stairs within the supported area.
- A bounded interpreter for a subset of the game's own event bytecode, plus text decoding. This does not execute GB instructions.
- Original initial event flags, readable signs, bedroom radio, bookshelves and supported conversations. Unknown interactions abort without committing partial changes.
- Native DOM dialogue, name/avatar choice, touch/keyboard controls and pause/resume.
- Native JSON state inside the backup envelope, separate IndexedDB storage and a separate tab lock. No emulator snapshot or cartridge save migration is implied.
- Synthetic tests, local-ROM tests, browser checks, and comparisons against the original decompression routine running in the reference emulator.
- Native production output excludes the emulator and original game assets.
This is **exploration**, not a complete new game or story-compatible save. Opening scenes, automatic map callbacks/coordinate events, story progression, NPC movement, decorations, animated terrain, full collision behavior, connected routes, party/inventory, battles and audio remain unported. Online Link and real-world steps come later.
## 1. Complete the overworld foundation
Extend extraction to every map. Read the ROM's map index and location names rather than shipping a copied world database. Implement connected borders, seamless transitions, dynamic warps, elevators, ledges, ice, currents, forced movement, surfing, cycling, cut trees and boulders.
Port callbacks, decorations, tile changes, scheduled NPC visibility, NPC movement, trainer sight lines and scripted movement. Complete terrain animation, layering and special palettes. Record deviations from the original until a comparison test closes them.
Acceptance: representative outdoor/interior/cave/connected-map traces produce matching positions, collisions, transitions and event changes in native and reference engines. Do not enable an area merely because its image can be decoded.
## 2. Opening story and first battle slice
Introduce native player, party, Pokémon, inventory, money and quest models. Read species, moves, growth curves, evolutions, encounters, trainers and items through typed ROM decoders. Define serialization/migrations before persisting new systems.
Expand script calls, choices, scene state, inventory commands and native special handlers. Translate routines reached through assembly calls into explicit native services; never fall back to emulator execution in the shipping game.
Implement the introduction, naming/clock setup, meeting Mom, Elm's starter selection, Route 29, Cherrygrove and the Mystery Egg round trip. Keep exploration saves separate until migration is meaningful.
Build the required native battle engine: deterministic randomness, stats, move selection, accuracy/critical checks, damage, status, turn order, switching, fainting, experience, capture and rewards. Compare controlled seeds and states with the original, including integer rounding and overflow behavior.
Acceptance: finish the opening quest, including encounters and the rival battle, party/inventory changes and save/reload, without emulation or manually setting story flags.
## 3. Complete Crystal
Port remaining battle commands, move/item effects, AI, special battles, evolutions, breeding, happiness, eggs and PC storage. Gen II behavior is the baseline; list intentional changes separately.
Complete field moves, gyms, Rocket events, phone/Pokégear, shops, healing, puzzles, minigames, daily events, roaming Pokémon and postgame. Cover Johto, the League, Kanto and the final battle. Track unsupported script/special handlers and add fixtures before translating each one.
Port the music/sound sequencer and synthesize its channels through Web Audio, reading music, effects and wave data from the local ROM. No recordings or CPU emulator are needed in the release. Verify timing and audio unlocking on phones.
Acceptance: main-game/postgame playthroughs, representative optional content, battle regression vectors, long sessions, clock rollover, interruption and save recovery. No unknown command is silently skipped. The full port is not complete until these checks pass.
## 4. Modern mobile presentation and delivery
Build touch interfaces directly over native models: party, pack, battle commands, PC and Pokégear. Keep UI separate from gameplay calculations. Add scalable text, configurable controls and accessible focus/announcements.
Add PWA installation/offline caching of the app shell only. ROM/data retention remains a separate explicit local preference; otherwise reselect the ROM after reopening. Add safe application updates, schema migrations, storage persistence/quota handling and portable backups. Investigate original `.sav` import only with verified structure, checksums and RTC conversion.
Acceptance: installation, offline startup, portrait/landscape, touch/audio, background/resume and updates on physical iPhone/Safari and Android/Chrome. Browser automation does not establish physical-device compatibility.
## 5. Internet Link
Implement a game-level protocol over native state. This architecture does not need to tunnel serial timing or synchronize GB CPU cycles.
Separate `LinkTransport`, `TradeSession` and `BattleSession`. Start with local peers, then private invite rooms using signaling and WebRTC data channels or a relay. Negotiate compatible schema/game versions and exchange only required game data. Never transmit ROMs or extracted assets.
Trades need durable preparation, a transaction ID, agreed completion, idempotent recovery and pre-trade backups for both players. Do not retry ambiguous completion automatically. Battles need deterministic state, agreed randomness, turn commands, state checks and a disconnect policy. Define validation, room expiry and persistence before public service access.
Acceptance: cross-network trades/battles and induced duplicate messages, latency, disconnects and reconnects at every phase, without duplication or lost Pokémon.
## 6. Walking companion
Separate a platform `StepProvider`, local activity ledger and reward rules. Keep real-world steps distinct from the in-game tile counter. Deduplicate history, record credited intervals, bound invalid data and handle denied/revoked permission.
Foreground browser motion can help prototype the experience; reliable pocket/background tracking needs platform support. Evaluate a native wrapper with iOS Core Motion and Android Health Connect/step sensors, checking current permissions and hardware availability when implementing it.
Choose the initial reward explicitly, such as companion progress or an item goal. Apply it through native game commands at safe points with a durable, idempotent ledger transaction. Sensor callbacks must not write directly into game state. Keep activity data on device by default.
Acceptance: a real walk with a locked phone is recorded, credited exactly once after reopening, and safely reflected in the game. Verify interaction with trades and save restoration.
## Working method
For each subsystem: identify PRET routines/data, implement native code behind a narrow interface, compare controlled inputs against the reference, then integrate a playable scenario. Retain synthetic fixtures for public CI. Keep real-ROM state, graphics, audio, screenshots and reference builds in ignored local locations.
See [NATIVE_PORT.md](NATIVE_PORT.md) for the current routine map and verification commands.