Commit initial
This commit is contained in:
@@ -0,0 +1,594 @@
|
||||
# Phase 2: Driver Management - Research
|
||||
|
||||
**Researched:** 2026-04-10
|
||||
**Domain:** ZIP upload handling, Windows INF parsing, content-addressed storage, HTMX-driven UI
|
||||
**Confidence:** HIGH (FastAPI upload patterns, Python stdlib zipfile/configparser, HTMX encoding), MEDIUM (INF encoding edge-cases)
|
||||
|
||||
---
|
||||
|
||||
<phase_requirements>
|
||||
## Phase Requirements
|
||||
|
||||
| ID | Description | Research Support |
|
||||
|----|-------------|-----------------|
|
||||
| DRV-01 | User can upload a driver package (ZIP containing INF + supporting files) | FastAPI `UploadFile` + `python-multipart`; `zipfile.ZipFile(io.BytesIO(...))` in-memory extraction |
|
||||
| DRV-02 | System parses uploaded INF files and extracts valid driver names (DriverDesc) | Python `configparser` reading `[Manufacturer]` → Models sections; `[Strings]` token resolution; encoding auto-detect (ANSI / UTF-8 / UTF-16 LE) |
|
||||
| DRV-03 | User can select driver name from parsed INF dropdown (no free-text) | HTMX `hx-post` + `hx-encoding="multipart/form-data"` + `hx-target` swap returning `<select>` fragment from server |
|
||||
| DRV-04 | Driver packages are persisted on Docker volume across container restarts | `DriverStore` (SHA256 content-addressed) already built in Phase 1; `Driver` ORM model already in schema |
|
||||
| DRV-05 | System flags unused files in driver packages to help reduce package size | Compare `zipfile.namelist()` against all `CopyFiles` / `SourceDisksFiles` references in the INF |
|
||||
</phase_requirements>
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
Phase 2 is three independent workstreams that converge into one UI flow: (1) a FastAPI endpoint that accepts a ZIP upload and stores it via the existing `DriverStore`, (2) a pure-Python INF parser that extracts driver names (`DriverDesc`) and detects unused files, and (3) an HTMX-powered Drivers page that shows an upload form and, after upload, replaces a placeholder with a populated `<select>` dropdown.
|
||||
|
||||
The storage infrastructure was completed in Phase 1. `DriverStore.save(data) -> sha256` and the `Driver` ORM model (with `driver_desc`, `inf_filename`, `architecture`, `has_cat_file` fields) are already in place. Phase 2 only needs to fill those fields by parsing the INF and register the driver record in SQLite.
|
||||
|
||||
INF parsing is the trickiest piece. Windows INF files use an INI-like format but have encoding variability (ANSI, UTF-8 with BOM, UTF-16 LE with BOM — all seen in real HP/Canon/Ricoh packages). Driver names (`DriverDesc`) are the left-hand values in the `[Models]` sections (e.g., `[Manufacturer.NTamd64]`), and they are frequently `%TOKEN%` references that must be resolved from the `[Strings]` section. Python's `configparser` handles this INI-like format well but needs an encoding sniff step and a `%`-token expander. Multi-model INF files (one INF with NTamd64 + NTarm64 + undecorated sections) must be deduplicated — extract all driver names, unique them, and present the merged list.
|
||||
|
||||
**Primary recommendation:** Use `configparser` with encoding auto-detection + a custom `%TOKEN%` resolver to extract DriverDesc values. Do NOT import the third-party `pyinf` library — its scope is too narrow and adds a dependency with no maintenance signal. All required INF parsing logic is achievable in ~60 lines of stdlib Python.
|
||||
|
||||
---
|
||||
|
||||
## Standard Stack
|
||||
|
||||
### Core (all already in requirements.txt from Phase 1)
|
||||
|
||||
| Library | Version | Purpose | Why Standard |
|
||||
|---------|---------|---------|--------------|
|
||||
| FastAPI | 0.115.x | Upload endpoint, HTMX fragment responses | Already installed; `UploadFile` built in |
|
||||
| python-multipart | 0.0.9 | Required by FastAPI for `UploadFile` | Already installed |
|
||||
| Peewee | 3.17.x | ORM — `Driver` record create/update | Already installed; schema already has all Phase 2 fields |
|
||||
| Jinja2 | 3.1.x | Render drivers page + HTMX partial (driver list fragment) | Already installed |
|
||||
| Python stdlib `zipfile` | 3.12 | Extract ZIP in-memory, list members | No new dependency |
|
||||
| Python stdlib `configparser` | 3.12 | Parse INF (INI-like format) | No new dependency |
|
||||
| Python stdlib `io` | 3.12 | `io.BytesIO` for in-memory ZIP | No new dependency |
|
||||
|
||||
### No New Dependencies Required
|
||||
|
||||
Phase 2 introduces zero new pip packages. Everything needed is either already installed (FastAPI/Peewee/Jinja2) or in the Python 3.12 stdlib (zipfile, configparser, io, hashlib).
|
||||
|
||||
### Alternatives Considered
|
||||
|
||||
| Instead of | Could Use | Tradeoff |
|
||||
|------------|-----------|----------|
|
||||
| `configparser` + custom token resolver | `pyinf` (third-party) | `pyinf` is a rudimentary single-developer project with no recent activity; stdlib configparser handles the INI format correctly with 30 extra lines of token resolution |
|
||||
| In-memory `io.BytesIO` extraction | `NamedTemporaryFile` on disk | In-memory is simpler for small driver ZIPs (< 50 MB typical); avoids temp file cleanup; adequate for this use case |
|
||||
|
||||
---
|
||||
|
||||
## Architecture Patterns
|
||||
|
||||
### Recommended File Layout for Phase 2
|
||||
|
||||
```
|
||||
imptune/
|
||||
├── api/
|
||||
│ ├── pages.py # add GET /drivers page route
|
||||
│ └── drivers.py # NEW: POST /drivers/upload endpoint
|
||||
├── services/
|
||||
│ └── inf_parser.py # NEW: parse_inf(zip_bytes) -> ParsedInf dataclass
|
||||
├── templates/
|
||||
│ ├── drivers.html # NEW: Drivers page (upload form + driver table)
|
||||
│ └── partials/
|
||||
│ └── driver_select.html # NEW: HTMX partial — <select> fragment
|
||||
```
|
||||
|
||||
### Pattern 1: INF Encoding Auto-Detection
|
||||
|
||||
**What:** INF files from real vendors arrive as ANSI (cp1252), UTF-8 with BOM, or UTF-16 LE with BOM. Attempting to read a UTF-16 file as UTF-8 raises `UnicodeDecodeError`. Sniff the BOM before parsing.
|
||||
|
||||
**When to use:** Always — called at the top of `parse_inf()`.
|
||||
|
||||
```python
|
||||
def _detect_encoding(raw: bytes) -> str:
|
||||
"""Detect INF file encoding from BOM bytes.
|
||||
Source: Microsoft docs — general-syntax-rules-for-inf-files
|
||||
(INF files may be ASCII/ANSI, UTF-8, or UTF-16 Unicode)
|
||||
"""
|
||||
if raw[:2] in (b'\xff\xfe', b'\xfe\xff'):
|
||||
return 'utf-16' # UTF-16 with BOM (LE or BE)
|
||||
if raw[:3] == b'\xef\xbb\xbf':
|
||||
return 'utf-8-sig' # UTF-8 with BOM
|
||||
return 'cp1252' # ANSI / Windows-1252 (safe fallback for Latin chars)
|
||||
```
|
||||
|
||||
### Pattern 2: INF DriverDesc Extraction via configparser + Token Resolution
|
||||
|
||||
**What:** The `[Manufacturer]` section lists manufacturer keys. Each key points to one or more `[ModelsSectionName]` or `[ModelsSectionName.NTamd64]` sections. Each entry in a Models section has the form:
|
||||
```
|
||||
device-description = install-section-name, hw-id[, compatible-id...]
|
||||
```
|
||||
The `device-description` (the left side) is the `DriverDesc` — the human-readable driver name. It is often a `%TOKEN%` reference that must be resolved from `[Strings]`.
|
||||
|
||||
**When to use:** Core of `parse_inf()`.
|
||||
|
||||
```python
|
||||
import configparser
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
@dataclass
|
||||
class ParsedInf:
|
||||
driver_names: list[str] # resolved DriverDesc values, deduplicated
|
||||
inf_filename: str # which .inf file inside the ZIP
|
||||
architecture: str | None # 'x64', 'x86', 'arm64', or None if ambiguous
|
||||
has_cat_file: bool # whether a .cat file exists in the ZIP
|
||||
unused_files: list[str] # ZIP members not referenced by the INF
|
||||
|
||||
def _resolve_tokens(value: str, strings: dict[str, str]) -> str:
|
||||
"""Expand %TOKEN% placeholders using the [Strings] section.
|
||||
Source: Microsoft docs — general-syntax-rules-for-inf-files
|
||||
"""
|
||||
def replacer(match):
|
||||
key = match.group(1).lower()
|
||||
return strings.get(key, match.group(0))
|
||||
return re.sub(r'%([^%]+)%', replacer, value)
|
||||
|
||||
def parse_inf(inf_text: str, inf_filename: str, zip_names: list[str]) -> ParsedInf:
|
||||
"""Parse a Windows INF file and extract driver names and metadata."""
|
||||
parser = configparser.RawConfigParser(
|
||||
comment_prefixes=(';', '#'),
|
||||
strict=False, # allow duplicate keys (real INFs have them)
|
||||
delimiters=('=',),
|
||||
)
|
||||
parser.read_string(inf_text)
|
||||
|
||||
# Build strings lookup (case-insensitive keys)
|
||||
strings: dict[str, str] = {}
|
||||
if parser.has_section('Strings'):
|
||||
for key, val in parser.items('Strings'):
|
||||
# Values may be quoted — strip surrounding quotes
|
||||
strings[key.lower()] = val.strip('"')
|
||||
|
||||
# Find all Models sections (decorated and undecorated)
|
||||
# Models sections are referenced from [Manufacturer] values
|
||||
driver_names: set[str] = set()
|
||||
arch_hints: set[str] = set()
|
||||
|
||||
# Gather manufacturer section names from [Manufacturer]
|
||||
models_section_names: list[str] = []
|
||||
if parser.has_section('Manufacturer'):
|
||||
for _mfg_key, mfg_val in parser.items('Manufacturer'):
|
||||
# Format: ModelsSection[,TargetOS,TargetOS...]
|
||||
parts = [p.strip() for p in mfg_val.split(',')]
|
||||
models_section_names.append(parts[0])
|
||||
|
||||
# For each referenced Models section (and its NTamd64/NTarm64 variants)
|
||||
for base_name in models_section_names:
|
||||
for section in parser.sections():
|
||||
# Match base name, base.NTamd64, base.NTarm64, base.NTx86, etc.
|
||||
if section.lower() == base_name.lower() or \
|
||||
section.lower().startswith(base_name.lower() + '.nt'):
|
||||
# Detect architecture from decoration
|
||||
suffix = section[len(base_name):].lower()
|
||||
if 'amd64' in suffix:
|
||||
arch_hints.add('x64')
|
||||
elif 'arm64' in suffix:
|
||||
arch_hints.add('arm64')
|
||||
elif 'x86' in suffix or suffix == '':
|
||||
arch_hints.add('x86')
|
||||
|
||||
for key, _val in parser.items(section):
|
||||
# key is the device-description (DriverDesc)
|
||||
resolved = _resolve_tokens(key, strings)
|
||||
# Filter out empty or purely numeric entries
|
||||
if resolved and not resolved.isdigit():
|
||||
driver_names.add(resolved)
|
||||
|
||||
arch = arch_hints.pop() if len(arch_hints) == 1 else None
|
||||
|
||||
# Detect .cat file
|
||||
has_cat = any(n.lower().endswith('.cat') for n in zip_names)
|
||||
|
||||
# Find unused files (not referenced anywhere in the INF text)
|
||||
inf_lower = inf_text.lower()
|
||||
unused: list[str] = []
|
||||
for member in zip_names:
|
||||
basename = member.split('/')[-1].split('\\')[-1]
|
||||
if basename.lower() not in inf_lower:
|
||||
unused.append(member)
|
||||
|
||||
return ParsedInf(
|
||||
driver_names=sorted(driver_names),
|
||||
inf_filename=inf_filename,
|
||||
architecture=arch,
|
||||
has_cat_file=has_cat,
|
||||
unused_files=unused,
|
||||
)
|
||||
```
|
||||
|
||||
### Pattern 3: FastAPI Upload Endpoint (sync handler, DriverStore + Peewee)
|
||||
|
||||
**What:** Receive a ZIP via `UploadFile`, store via `DriverStore`, parse the INF, write the `Driver` record.
|
||||
|
||||
**When to use:** `POST /drivers/upload` — called by the HTMX form.
|
||||
|
||||
```python
|
||||
# imptune/api/drivers.py
|
||||
from fastapi import APIRouter, UploadFile, File, HTTPException, Request
|
||||
from fastapi.responses import HTMLResponse
|
||||
from fastapi.templating import Jinja2Templates
|
||||
import zipfile, io
|
||||
from pathlib import Path
|
||||
|
||||
from imptune.config import DRIVERS_DIR
|
||||
from imptune.storage.driver_store import DriverStore
|
||||
from imptune.db.models import Driver
|
||||
from imptune.services.inf_parser import parse_inf, _detect_encoding
|
||||
|
||||
router = APIRouter(prefix="/drivers")
|
||||
templates = Jinja2Templates(directory=str(Path(__file__).parent.parent / "templates"))
|
||||
|
||||
MAX_UPLOAD_BYTES = 100 * 1024 * 1024 # 100 MB — generous for printer driver ZIPs
|
||||
|
||||
@router.post("/upload", response_class=HTMLResponse)
|
||||
def upload_driver(request: Request, file: UploadFile = File(...)):
|
||||
"""Accept ZIP upload, parse INF, store driver, return HTMX partial."""
|
||||
data = file.file.read()
|
||||
if len(data) > MAX_UPLOAD_BYTES:
|
||||
raise HTTPException(413, "Driver package exceeds 100 MB limit")
|
||||
if not file.filename.lower().endswith('.zip'):
|
||||
raise HTTPException(400, "Only ZIP files are accepted")
|
||||
|
||||
# Validate it's actually a ZIP
|
||||
if not zipfile.is_zipfile(io.BytesIO(data)):
|
||||
raise HTTPException(400, "Uploaded file is not a valid ZIP archive")
|
||||
|
||||
with zipfile.ZipFile(io.BytesIO(data)) as zf:
|
||||
names = zf.namelist()
|
||||
# Security: reject zip-slip paths
|
||||
for name in names:
|
||||
if name.startswith('/') or '..' in name:
|
||||
raise HTTPException(400, f"Dangerous path in ZIP: {name}")
|
||||
# Find INF file(s)
|
||||
inf_names = [n for n in names if n.lower().endswith('.inf')]
|
||||
if not inf_names:
|
||||
raise HTTPException(400, "No .inf file found in ZIP")
|
||||
# Use the first INF (most driver ZIPs have exactly one)
|
||||
inf_bytes = zf.read(inf_names[0])
|
||||
|
||||
encoding = _detect_encoding(inf_bytes)
|
||||
inf_text = inf_bytes.decode(encoding, errors='replace')
|
||||
parsed = parse_inf(inf_text, inf_names[0], names)
|
||||
|
||||
# Store file (SHA256 content-addressed, deduplication built in)
|
||||
store = DriverStore(DRIVERS_DIR)
|
||||
sha256 = store.save(data)
|
||||
|
||||
# Upsert Driver record (idempotent on sha256)
|
||||
driver, _created = Driver.get_or_create(
|
||||
sha256=sha256,
|
||||
defaults={
|
||||
'original_filename': file.filename,
|
||||
'size_bytes': len(data),
|
||||
'driver_desc': parsed.driver_names[0] if parsed.driver_names else None,
|
||||
'inf_filename': parsed.inf_filename,
|
||||
'architecture': parsed.architecture,
|
||||
'has_cat_file': parsed.has_cat_file,
|
||||
}
|
||||
)
|
||||
|
||||
# Return HTMX partial: driver list fragment
|
||||
drivers = list(Driver.select().order_by(Driver.uploaded_at.desc()))
|
||||
return templates.TemplateResponse(
|
||||
request=request,
|
||||
name="partials/driver_list.html",
|
||||
context={
|
||||
"drivers": drivers,
|
||||
"new_driver": driver,
|
||||
"parsed": parsed,
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
### Pattern 4: HTMX Upload Form with Target Swap
|
||||
|
||||
**What:** The upload form posts to `/drivers/upload` and replaces the `#driver-list` section with the returned HTML fragment.
|
||||
|
||||
**When to use:** Drivers page — upload form section.
|
||||
|
||||
```html
|
||||
<!-- templates/drivers.html (relevant fragment) -->
|
||||
<form
|
||||
hx-post="/drivers/upload"
|
||||
hx-encoding="multipart/form-data"
|
||||
hx-target="#driver-list"
|
||||
hx-swap="outerHTML"
|
||||
hx-indicator="#upload-spinner"
|
||||
>
|
||||
<label for="driver-file">Driver Package (ZIP)</label>
|
||||
<input type="file" id="driver-file" name="file" accept=".zip" required>
|
||||
<button type="submit">Upload</button>
|
||||
<span id="upload-spinner" class="htmx-indicator">Uploading...</span>
|
||||
</form>
|
||||
|
||||
<div id="driver-list">
|
||||
{% include "partials/driver_list.html" %}
|
||||
</div>
|
||||
```
|
||||
|
||||
The server returns a replacement `<div id="driver-list">...</div>` containing the updated driver table plus any success/warning messages (unused files count, architecture, etc.).
|
||||
|
||||
### Anti-Patterns to Avoid
|
||||
|
||||
- **`zipfile.extractall()` without path validation:** Vulnerable to Zip Slip. Always iterate `zf.namelist()` and reject entries with `..` or absolute paths before reading.
|
||||
- **`configparser` with `strict=True` for INF files:** Real INF files frequently contain duplicate keys across sections (multiple models with similar names). `strict=False` is required.
|
||||
- **Using configparser's interpolation for `%TOKEN%` expansion:** configparser's built-in interpolation uses `%(key)s` syntax, not `%KEY%`. Use `RawConfigParser` and a separate regex-based `_resolve_tokens()` function.
|
||||
- **Assuming one INF per ZIP:** Some vendor packages contain multiple INF files (x64 + x86 in different subdirectories). Parse the first `.inf` found; flag if multiples exist.
|
||||
- **`async def` route for upload:** Since `DriverStore.save()` and `Driver.get_or_create()` are synchronous (Peewee), use a regular `def` route. FastAPI runs sync handlers in a thread pool automatically — no blocking.
|
||||
- **Storing parsed driver names as a list in SQLite:** The existing `Driver.driver_desc` is a single `CharField`. For Phase 2, store the first (or primary) driver name. If multi-driver-name support is needed, that is a Phase 2+ schema change — but the current schema supports the DRV-03 requirement of a single dropdown choice per uploaded package.
|
||||
|
||||
---
|
||||
|
||||
## Don't Hand-Roll
|
||||
|
||||
| Problem | Don't Build | Use Instead | Why |
|
||||
|---------|-------------|-------------|-----|
|
||||
| INF file parsing | Custom INI tokenizer from scratch | `configparser.RawConfigParser` + `_resolve_tokens()` | INI format edge cases: duplicate keys, inline comments, continuation lines, quoted strings |
|
||||
| ZIP extraction | Custom byte-level ZIP reader | `zipfile.ZipFile(io.BytesIO(data))` | Handles all ZIP variants (ZIP64, deflate, stored); stdlib, no extra dep |
|
||||
| Content-addressed file storage | New storage abstraction | `DriverStore` (already built in Phase 1) | SHA256 + dedup already implemented and tested |
|
||||
| Driver ORM record | Raw SQL INSERT | `Driver.get_or_create(sha256=sha256, ...)` | Idempotent on re-upload; schema already has all Phase 2 fields |
|
||||
| HTMX multipart upload form | Custom `fetch()` JavaScript | `hx-encoding="multipart/form-data"` on `<form>` | One attribute handles encoding; HTMX manages request + swap; no JS needed |
|
||||
|
||||
**Key insight:** Phase 1 already solved persistence and deduplication. Phase 2's only novel logic is the INF parser and the upload endpoint wiring.
|
||||
|
||||
---
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Pitfall 1: INF Encoding Not Detected — `UnicodeDecodeError`
|
||||
|
||||
**What goes wrong:** Reading a UTF-16 LE INF file as UTF-8 raises `UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0`. Typical for HP and Canon INF files, which ship as UTF-16 LE with BOM.
|
||||
|
||||
**Why it happens:** Windows uses UTF-16 internally; INF files signed for x64 are frequently written in UTF-16.
|
||||
|
||||
**How to avoid:** Always sniff the first 3 bytes before calling `decode()`. UTF-16 LE BOM is `\xff\xfe`; UTF-16 BE BOM is `\xfe\xff`; UTF-8 BOM is `\xef\xbb\xbf`. Fall back to `cp1252` (not `utf-8`) for BOM-less files — `cp1252` is a superset of Latin-1 and handles Western European printer names without errors.
|
||||
|
||||
**Warning signs:** `UnicodeDecodeError` on real vendor ZIP uploads.
|
||||
|
||||
### Pitfall 2: `configparser` `strict=True` Fails on Duplicate Keys
|
||||
|
||||
**What goes wrong:** `configparser.DuplicateOptionError` is raised when parsing INF files that list multiple models with the same base name but different decorations.
|
||||
|
||||
**Why it happens:** `configparser` defaults to `strict=True` which rejects duplicate keys within the same section. INF files frequently have this structure.
|
||||
|
||||
**How to avoid:** Instantiate with `configparser.RawConfigParser(strict=False)`.
|
||||
|
||||
### Pitfall 3: `%TOKEN%` Values Appear Literally in Dropdown
|
||||
|
||||
**What goes wrong:** Driver dropdown shows `%HP_LASERJET_P2055D%` instead of `HP LaserJet P2055d`.
|
||||
|
||||
**Why it happens:** `configparser` does not process `%...%` INF token syntax — only its own `%(key)s` interpolation, which is irrelevant here.
|
||||
|
||||
**How to avoid:** After parsing, apply `_resolve_tokens(raw_value, strings_dict)` to every extracted device-description. Build the `strings` dict from `[Strings]` section values (stripped of surrounding double-quotes).
|
||||
|
||||
### Pitfall 4: Zip Slip Vulnerability on Upload
|
||||
|
||||
**What goes wrong:** A malicious or malformed ZIP contains entries like `../../etc/passwd`. `zipfile.extractall()` writes those files to the host filesystem.
|
||||
|
||||
**Why it happens:** The default Python `zipfile.extractall()` does not check for traversal paths.
|
||||
|
||||
**How to avoid:** Never call `extractall()`. Read individual members with `zf.read(name)` after validating each `name` in `zf.namelist()` does not start with `/` or contain `..`. This is safe because the file is read into memory, not extracted to disk.
|
||||
|
||||
**Warning signs:** Any code that calls `zf.extractall(path)` without path filtering.
|
||||
|
||||
### Pitfall 5: No `.inf` in ZIP Returns a Confusing Error
|
||||
|
||||
**What goes wrong:** Technician uploads a ZIP that contains only drivers but not the INF (common when someone zips the wrong folder). The endpoint crashes with a `KeyError` or returns HTTP 500.
|
||||
|
||||
**Why it happens:** Code assumes at least one `.inf` member exists.
|
||||
|
||||
**How to avoid:** Explicit check: `if not inf_names: raise HTTPException(400, "No .inf file found in ZIP")`. Return the error as an HTMX response so it appears in-page without a full reload. Render the error inside the `#driver-list` target.
|
||||
|
||||
### Pitfall 6: Multi-INF ZIP — Wrong Driver Names Selected
|
||||
|
||||
**What goes wrong:** A ZIP with both x64 and x86 INF files (in subdirectories) produces duplicate driver names if both are parsed, or wrong names if the wrong file is parsed.
|
||||
|
||||
**Why it happens:** Some vendors ship a ZIP with `x64/printer.inf` and `x86/printer.inf` containing different model lists.
|
||||
|
||||
**How to avoid:** Prefer INF files whose path does not contain `x86` when an `amd64`/`x64` sibling exists. Sort candidates to prefer `amd64`/`NTamd64` variants. Log a warning when multiple INF files are found; surface the INF filename in the UI so the technician can verify.
|
||||
|
||||
### Pitfall 7: `Driver.driver_desc` Stores Only One Name (Schema Limitation)
|
||||
|
||||
**What goes wrong:** An INF file contains 15 different models. Only one is stored in `driver_desc`. DRV-03 requires a dropdown of all parsed names — but after page reload, only the stored name is shown.
|
||||
|
||||
**Why it happens:** The Phase 1 schema stores a single `driver_desc` CharField.
|
||||
|
||||
**How to avoid:** Store all parsed driver names as a JSON-encoded list in `driver_desc` (e.g., `json.dumps(driver_names)`). The dropdown is generated by parsing the stored JSON. Alternatively, store the INF text itself. The simplest solution that satisfies DRV-03 without schema changes: store `json.dumps(parsed.driver_names)` in `driver_desc` and decode at read time. This fits in one CharField with no migration.
|
||||
|
||||
---
|
||||
|
||||
## Code Examples
|
||||
|
||||
### INF Encoding Detection
|
||||
|
||||
```python
|
||||
# imptune/services/inf_parser.py
|
||||
def _detect_encoding(raw: bytes) -> str:
|
||||
"""Sniff BOM bytes to determine INF file encoding.
|
||||
Source: Microsoft WDK — general-syntax-rules-for-inf-files
|
||||
"""
|
||||
if raw[:2] in (b'\xff\xfe', b'\xfe\xff'):
|
||||
return 'utf-16'
|
||||
if raw[:3] == b'\xef\xbb\xbf':
|
||||
return 'utf-8-sig'
|
||||
return 'cp1252'
|
||||
```
|
||||
|
||||
### Safe ZIP Member Reading (Zip Slip Prevention)
|
||||
|
||||
```python
|
||||
# Before reading any member:
|
||||
for name in zf.namelist():
|
||||
if name.startswith('/') or '..' in name:
|
||||
raise HTTPException(400, f"Dangerous path in ZIP: {name}")
|
||||
# Then read safely:
|
||||
inf_bytes = zf.read(inf_names[0])
|
||||
```
|
||||
|
||||
### HTMX Upload Form (multipart)
|
||||
|
||||
```html
|
||||
<!-- hx-encoding is the critical attribute for file upload -->
|
||||
<form
|
||||
hx-post="/drivers/upload"
|
||||
hx-encoding="multipart/form-data"
|
||||
hx-target="#driver-list"
|
||||
hx-swap="outerHTML"
|
||||
>
|
||||
<input type="file" name="file" accept=".zip" required>
|
||||
<button type="submit">Upload Driver Package</button>
|
||||
</form>
|
||||
```
|
||||
|
||||
### Driver Record Upsert (idempotent on SHA256)
|
||||
|
||||
```python
|
||||
# Peewee get_or_create — safe for duplicate uploads
|
||||
driver, created = Driver.get_or_create(
|
||||
sha256=sha256,
|
||||
defaults={
|
||||
'original_filename': file.filename,
|
||||
'size_bytes': len(data),
|
||||
'driver_desc': json.dumps(parsed.driver_names),
|
||||
'inf_filename': parsed.inf_filename,
|
||||
'architecture': parsed.architecture,
|
||||
'has_cat_file': parsed.has_cat_file,
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### Unused Files Detection
|
||||
|
||||
```python
|
||||
# Compare ZIP member basenames against INF text
|
||||
# Source: DRV-05 requirement
|
||||
inf_lower = inf_text.lower()
|
||||
unused = []
|
||||
for member in zip_names:
|
||||
basename = member.rsplit('/', 1)[-1].rsplit('\\', 1)[-1]
|
||||
if basename.lower() not in inf_lower:
|
||||
unused.append(member)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## State of the Art
|
||||
|
||||
| Old Approach | Current Approach | When Changed | Impact |
|
||||
|--------------|------------------|--------------|--------|
|
||||
| Free-text driver name entry | Dropdown from parsed INF DriverDesc | DRV-03 (this phase) | Eliminates typos; driver name matches exactly what pnputil expects |
|
||||
| Manual INF file navigation | Automated DriverDesc extraction + `%TOKEN%` resolution | This phase | Technician never needs to open the INF |
|
||||
| `pyinf` third-party library | Python stdlib `configparser` + custom resolver | This phase decision | Zero new dependency; full control over edge-case handling |
|
||||
|
||||
**Note on driver name storage:** The `driver_desc` column was designed in Phase 1 as a single `CharField`. Storing `json.dumps(list)` is the correct approach to preserve all model names without a schema migration. The Phase 3 printer configuration form reads this JSON to render the `<select>` dropdown.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Multi-INF ZIPs: which INF wins?**
|
||||
- What we know: Some vendor packages (HP Universal Print Driver) contain multiple INF files for different architectures or print frameworks.
|
||||
- What's unclear: Whether to parse all INFs and merge names, or to pick one based on file path heuristics.
|
||||
- Recommendation: Parse the INF whose path contains `amd64` or `x64` if multiple exist; fall back to the first `.inf` alphabetically. Surface the selected INF filename in the UI and the driver list so the technician can verify.
|
||||
|
||||
2. **DriverDesc deduplication across Models sections**
|
||||
- What we know: When an INF has both `[Mfg.NTamd64]` and `[Mfg.NTarm64]` sections, the same `%TOKEN%` key appears in both. After token resolution, values are identical.
|
||||
- What's unclear: Whether any INFs intentionally list different names per architecture.
|
||||
- Recommendation: Use a `set()` for deduplication during extraction, then `sorted()` for consistent dropdown order.
|
||||
|
||||
3. **Unused-file detection accuracy**
|
||||
- What we know: The naive approach (check if basename appears in INF text) will produce false negatives for files referenced only by full path, and false positives for files referenced by registry entries or other INF directives not in `CopyFiles`.
|
||||
- What's unclear: How accurate DRV-05 needs to be — is a "best-effort" count acceptable?
|
||||
- Recommendation: Implement the naive basename-in-text approach for Phase 2. It handles the common case correctly (CopyFiles lists basenames). Mark it as best-effort in the UI: "X files may be unused". A more precise parser checking `CopyFiles` / `SourceDisksFiles` directives specifically is a Phase 2+ enhancement.
|
||||
|
||||
---
|
||||
|
||||
## Validation Architecture
|
||||
|
||||
### Test Framework
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Framework | pytest (already installed in requirements-dev.txt from Phase 1) |
|
||||
| Config file | None — uses pytest auto-discovery |
|
||||
| Quick run command | `pytest tests/ -x -q` |
|
||||
| Full suite command | `pytest tests/ -v` |
|
||||
|
||||
### Phase Requirements -> Test Map
|
||||
|
||||
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|
||||
|--------|----------|-----------|-------------------|-------------|
|
||||
| DRV-01 | POST /drivers/upload with valid ZIP returns 200 and HTML fragment | integration | `pytest tests/test_driver_upload.py::test_upload_valid_zip -x` | Wave 0 |
|
||||
| DRV-01 | POST /drivers/upload with non-ZIP file returns 400 | unit | `pytest tests/test_driver_upload.py::test_upload_non_zip -x` | Wave 0 |
|
||||
| DRV-01 | POST /drivers/upload with ZIP containing no INF returns 400 | unit | `pytest tests/test_driver_upload.py::test_upload_no_inf -x` | Wave 0 |
|
||||
| DRV-02 | parse_inf extracts DriverDesc from simple INF (token-free) | unit | `pytest tests/test_inf_parser.py::test_simple_driver_desc -x` | Wave 0 |
|
||||
| DRV-02 | parse_inf resolves %TOKEN% values from [Strings] section | unit | `pytest tests/test_inf_parser.py::test_token_resolution -x` | Wave 0 |
|
||||
| DRV-02 | parse_inf handles UTF-16 LE BOM encoded INF | unit | `pytest tests/test_inf_parser.py::test_utf16_encoding -x` | Wave 0 |
|
||||
| DRV-02 | parse_inf handles multi-model INF (NTamd64 + undecorated) | unit | `pytest tests/test_inf_parser.py::test_multi_model_inf -x` | Wave 0 |
|
||||
| DRV-03 | Drivers page (GET /drivers) renders upload form | integration | `pytest tests/test_driver_upload.py::test_drivers_page -x` | Wave 0 |
|
||||
| DRV-03 | Upload response contains populated `<select>` with driver names | integration | `pytest tests/test_driver_upload.py::test_upload_returns_select -x` | Wave 0 |
|
||||
| DRV-04 | Uploaded driver file exists on disk under DRIVERS_DIR after upload | integration | `pytest tests/test_driver_upload.py::test_driver_persisted -x` | Wave 0 |
|
||||
| DRV-04 | Re-uploading same ZIP does not create duplicate Driver record | integration | `pytest tests/test_driver_upload.py::test_dedup_upload -x` | Wave 0 |
|
||||
| DRV-05 | parse_inf returns unused_files list for files not in INF text | unit | `pytest tests/test_inf_parser.py::test_unused_files -x` | Wave 0 |
|
||||
| DRV-05 | Upload response includes unused-file count/list in HTML | integration | `pytest tests/test_driver_upload.py::test_unused_files_in_response -x` | Wave 0 |
|
||||
|
||||
### Sampling Rate
|
||||
|
||||
- **Per task commit:** `pytest tests/ -x -q`
|
||||
- **Per wave merge:** `pytest tests/ -v`
|
||||
- **Phase gate:** Full suite green before `/gsd:verify-work`
|
||||
|
||||
### Wave 0 Gaps
|
||||
|
||||
- [ ] `tests/test_inf_parser.py` — covers DRV-02 (INF parsing, token resolution, encoding, multi-model)
|
||||
- [ ] `tests/test_driver_upload.py` — covers DRV-01, DRV-03, DRV-04, DRV-05 (upload endpoint integration tests)
|
||||
- [ ] `tests/fixtures/sample.inf` — minimal valid INF fixture with `%TOKEN%` values
|
||||
- [ ] `tests/fixtures/sample_utf16.inf` — UTF-16 LE encoded INF fixture
|
||||
- [ ] `tests/fixtures/sample_multi_model.inf` — INF with NTamd64 and undecorated sections
|
||||
- [ ] `imptune/services/__init__.py` — package marker (services/ directory exists in project structure but is empty)
|
||||
|
||||
*(Framework already installed; conftest.py with `tmp_data_dir` fixture already covers DB isolation)*
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
### Primary (HIGH confidence)
|
||||
|
||||
- [Microsoft WDK — INF Models Section](https://learn.microsoft.com/en-us/windows-hardware/drivers/install/inf-models-section) — device-description = install-section-name,hw-id format; architecture decorations (NTamd64, NTarm64)
|
||||
- [Microsoft WDK — General Syntax Rules for INF Files](https://learn.microsoft.com/en-us/windows-hardware/drivers/install/general-syntax-rules-for-inf-files) — encoding (ANSI/UTF-8/UTF-16), `%strkey%` token format, comment syntax, case-insensitivity
|
||||
- [Microsoft WDK — Printer INF File Entries](https://learn.microsoft.com/en-us/windows-hardware/drivers/print/printer-inf-file-entries) — DriverFile, DataFile, ConfigFile, DriverDesc usage in Ntprint.dll
|
||||
- [Microsoft WDK — Decorations in Printer INF Files](https://learn.microsoft.com/en-us/windows-hardware/drivers/print/decorations-in-printer-inf-files) — NTamd64 decoration mandatory for x64 since WS2003 SP1
|
||||
- [FastAPI — Request Files](https://fastapi.tiangolo.com/tutorial/request-files/) — `UploadFile`, `File(...)`, reading file bytes
|
||||
- [HTMX — hx-encoding attribute](https://htmx.org/attributes/hx-encoding/) — `multipart/form-data` required for file uploads
|
||||
- [HTMX — File Upload example](https://htmx.org/examples/file-upload/) — progress tracking, server response swap
|
||||
- Python 3.12 stdlib `zipfile` — `ZipFile(io.BytesIO())`, `namelist()`, `read()`, `is_zipfile()`
|
||||
- Python 3.12 stdlib `configparser` — `RawConfigParser(strict=False)`, `read_string()`, `has_section()`, `items()`
|
||||
|
||||
### Secondary (MEDIUM confidence)
|
||||
|
||||
- [Snyk / Zip Slip Vulnerability](https://github.com/snyk/zip-slip-vulnerability) — path traversal attack pattern; prevention via `namelist()` validation
|
||||
- [FastAPI file size limiting discussion](https://github.com/fastapi/fastapi/issues/362) — post-read size check pattern; no built-in pre-rejection mechanism
|
||||
- [Microsoft WDK — Printer INF File Data Sections](https://learn.microsoft.com/en-us/windows-hardware/drivers/print/printer-inf-file-data-sections) — DataSection pattern; Previous Names section
|
||||
|
||||
### Tertiary (LOW confidence)
|
||||
|
||||
- [pyinf GitHub](https://github.com/tty72/pyinf) — reviewed and rejected: rudimentary, no recent activity, no benefit over stdlib
|
||||
|
||||
---
|
||||
|
||||
## Metadata
|
||||
|
||||
**Confidence breakdown:**
|
||||
- FastAPI upload patterns: HIGH — official docs verified
|
||||
- HTMX multipart form: HIGH — official docs verified
|
||||
- INF format (overall): HIGH — Microsoft WDK official docs
|
||||
- INF encoding handling (edge cases): MEDIUM — documented rule is clear but real-world INF corpus has variability; edge cases may surface during testing
|
||||
- Unused-file detection accuracy: MEDIUM — naive approach is best-effort; accuracy depends on INF structure
|
||||
|
||||
**Research date:** 2026-04-10
|
||||
**Valid until:** 2026-05-10 (stable ecosystem)
|
||||
Reference in New Issue
Block a user