An OEM structures a repeatable device security evaluation by asking the same questions of every supplier and every model, in three steps. First, define the programme: which models, chipsets and firmware are in scope, and who holds build access and signing authority. Second, require evidence: trust boundaries, key ownership and how each control was validated on your devices. Third, scope the lifecycle: who maintains what, for how long, and at what recurring effort. This guide gives a checklist for each step, and shows how Otukan answers it.
Define the programme
An evaluation is only repeatable if its scope is written down first. Record the answers per model. The questions assume devices whose firmware the OEM or ODM builds, which in practice means Android-based devices; Apple builds and signs the firmware for iOS devices.
Devices and firmware
- Which device models are in scope, and which system-on-chip each uses?
- Which operating system and version does each model ship with, and which versions will it receive?
- Which board support package and vendor firmware versions are in use?
- How many models share one firmware baseline?
- Which hardware security features does each chipset provide: trusted execution environment, secure element, hardware-backed key store, attestation keys provisioned at manufacture?
Integration authority
- Who builds the production system image: the OEM, the ODM or both?
- Who holds the keys for system images and over-the-air updates?
- What do the ODM and chipset vendor agreements allow a third party to change?
- Who approves a firmware release, and who can stop one?
- Where will integration work be done, and on whose build infrastructure?
Threats in scope
- Which threats is the programme meant to address: modified firmware, unlocked bootloaders, privilege escalation, data extraction from lost devices, supply chain changes?
- Which threats are explicitly out of scope?
- Who signs off the threat model on the OEM side?
How Otukan answers. Otukan starts every programme with a qualification stage that produces these answers as a written scope statement per model, and agrees the device threat model with your engineers before any integration work.
Require evidence
Ask each supplier for the same evidence, in the same form, so answers can be compared side by side.
Trust boundaries
- A diagram showing which layer each control runs in: boot chain, kernel, system services, apps or a server.
- For each control, which layer it trusts and what happens if that layer is compromised.
- Which signals are signed by a hardware-backed key, and which are reported by software.
- Which data leaves the device, where it goes and who can read it.
Key ownership
- A list of every key involved: boot, system image, update, attestation, service and data encryption.
- For each key: who generates it, where it is stored, who can use it and how it is rotated.
- What happens if a key is lost or exposed, and who acts.
- Confirmation that no supplier needs your signing keys outside your own signing process.
Validation
- Test results on the physical models in scope, not on reference boards alone.
- Tests of update, rollback, factory reset and repair paths.
- Behaviour when a check fails: what the device does, what is logged, who is told.
- How results can be reproduced by your own team or an independent tester.
- A clear statement of what each control addresses and where it stops.
| Evidence item | Strong answer | Weak answer |
|---|---|---|
| Model and chipset scope | Named per model, with firmware versions | “Android devices” in general |
| Hardware-backed features | Tied to a named chipset feature on your model | Assumed from the platform |
| Key custody | Documented per key, with owner and rotation | “Secure key management” |
| Control validation | Reproducible tests on your devices | Results from other devices, or no method |
| Scope of each control | Written for each control | Absent |
How Otukan answers. Otukan provides a trust-boundary diagram, a per-key custody list and test results on your physical models with a reproducible method. Otukan works inside your signing process, so your keys stay with you.
Scope the lifecycle
Device security runs for the supported life of each model. Document maintenance scope in the evaluation, and base any cost comparison on evidence rather than estimated savings.
Maintenance scope
- Who integrates platform and vendor security patches, and how are they retested?
- Who rebuilds and retests the protection for each firmware update?
- Who supports a new operating system version on an existing model?
- What is the end-of-support date per model, and what happens to devices still in use?
Cost drivers to record
Record each driver as scope and owner.
| Cost driver | What to record |
|---|---|
| Number of models and chipsets | Each adds integration and test work |
| Firmware update frequency | Each release needs rebuilding and retesting |
| Operating system upgrades | May need new integration work per model |
| Supplier coordination | ODM and chipset vendor effort, and who funds it |
| Key custody | Hardware, procedures and people over the whole life |
| Custom OS engineering | Work specific to one device, scoped separately from reusable controls |
Responsibility template
| Duty | OEM | ODM | Chipset vendor | Security supplier |
|---|---|---|---|---|
| Firmware build | ||||
| Release signing | ||||
| Security patches | ||||
| Control maintenance | ||||
| Key custody and rotation | ||||
| Incident response | ||||
| End of support |
Fill one copy per model. An empty cell is a decision still to make.
How Otukan answers. Otukan completes this template with you for every model, rebuilds and retests its protection with each firmware release, and separates reusable protection from custom OS engineering so recurring effort is visible from the start.
How Otukan fits OEM, institutional and operator programmes is covered on device security use cases. How integration proceeds once a supplier is chosen is covered on OEM device security integration. How to test a programme is covered on mobile device security evaluation. For the product overview, see mobile device security.