oaknut.hfe

HFE (HxC Floppy Emulator) flux images, read-only. An HFE file records each track as the bitstream the drive head delivers, so reading one is a pipeline: the container yields each track’s flux samples, the decoder finds the FM and MFM sectors on them, and flattening lays the standard sectors out as the sector image a filesystem reads.

You rarely need this package directly: it registers HFEDecoder on the oaknut.decoder axis, so reader_for(), and with it every disc command, opens an HFE file as its flattened sector image. Use it to look below the sector level, at the copy protection on an original disc for instance.

>>> from oaknut.hfe import HFEImage, analyse, decode_image, flatten
>>> disc = decode_image(HFEImage.from_filepath(DUNJUNZ_S1_FILEPATH))
>>> flat = flatten(disc)
>>> flat.layout.num_tracks, flat.layout.sectors_per_track, flat.suffix
(40, 10, '.ssd')
>>> [(f.track, f.remarks) for f in analyse(disc, flat.layout) if f.anomalous]
[(40, ('unformatted',))]

Every name documented here is importable directly from oaknut.hfe.

The container

The HFE file container.

An HFE file starts with a 512-byte header, followed at track_list_offset * 512 by a track lookup table of four bytes per track: the track’s offset in 512-byte blocks and its length in bytes, both little-endian. A track’s length covers both sides, which are interleaved in 256-byte halves of each 512-byte block: the first half belongs to side 0, the second to side 1.

Every byte holds eight flux samples, least significant bit first, on a clock of twice the header’s bit rate: at 250 kbit/s a sample is 2 µs. A 1 sample is a flux transition. (So one FM half-cell spans two samples and one MFM half-cell spans one; see oaknut.hfe.ibm.)

Version 3 (HXCHFEV3) adds in-stream opcodes, recognised by a high nibble of 0xF after bit reversal, which cannot occur in valid FM or MFM data:

NOP

F0

no samples

SETINDEX

F1

the index pulse occurs here

SETBITRATE

F2

the next byte n sets the rate to 36 MHz / (2 n)

SKIPBITS

F3

the next byte n: skip the first n samples of the byte after it

RAND

F4

eight weak samples, which read differently each time

Version 1 (HXCPICFE) has no opcodes. Weak samples read here as zeros (no transition), and are counted in TrackSamples.weak_samples.

Ported from Beebium’s HfeFormatHandler.

class oaknut.hfe.HFEImage(data, header)

A parsed HFE file, decoding each track’s samples on demand.

Parameters:
classmethod from_filepath(filepath)

Read and parse the HFE file at filepath.

Parameters:

filepath (Path)

Return type:

HFEImage

classmethod from_bytes(data)

Parse an HFE file held in data.

Raises:

HFEFormatError – The header is not a valid HFE v1 or v3 header.

Parameters:

data (bytes)

Return type:

HFEImage

track(track, side)

The samples of physical track on side.

Raises:
  • IndexError – track or side is outside the image.

  • HFEFormatError – The track’s data runs past the end of the file.

Parameters:
Return type:

TrackSamples

class oaknut.hfe.HFEHeader(version, num_tracks, num_sides, track_encoding, bitrate_kbps, rpm, interface_mode, track_list_offset, write_allowed, single_step)

The fields of an HFE file header.

Parameters:
track_encoding: TrackEncoding | int

A TrackEncoding, or the raw byte when it is not one.

single_step: bool

False when the image is meant for a double-stepping drive.

class oaknut.hfe.TrackSamples(samples, weak_samples=0, index_positions=(), bitrates_kbps=())

The flux samples of one side of one track, in time order.

Parameters:
samples: str

"0"/"1" samples; "1" is a flux transition.

weak_samples: int = 0

How many of samples came from weak (RAND) bytes.

index_positions: tuple[int, ...] = ()

Sample positions of SETINDEX opcodes.

bitrates_kbps: tuple[int, ...] = ()

Bit rates set by SETBITRATE opcodes, in kbit/s, in order.

class oaknut.hfe.TrackEncoding(*values)

The track-encoding hint in the HFE header.

Capture tools set it from what they expected, not what they found, so the decoder searches every track for both FM and MFM regardless.

ISOIBM_MFM = 0
AMIGA_MFM = 1
ISOIBM_FM = 2
EMU_FM = 3
UNKNOWN = 255
oaknut.hfe.HFE_V1_SIGNATURE = b'HXCPICFE'

bytes(iterable_of_ints) -> bytes bytes(string, encoding[, errors]) -> bytes bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer bytes(int) -> bytes object of size given by the parameter initialized with null bytes bytes() -> empty bytes object

Construct an immutable array of bytes from:
  • an iterable yielding integers in range(256)

  • a text string encoded using the specified encoding

  • any object implementing the buffer API.

  • an integer

oaknut.hfe.HFE_V3_SIGNATURE = b'HXCHFEV3'

bytes(iterable_of_ints) -> bytes bytes(string, encoding[, errors]) -> bytes bytes(bytes_or_buffer) -> immutable copy of bytes_or_buffer bytes(int) -> bytes object of size given by the parameter initialized with null bytes bytes() -> empty bytes object

Construct an immutable array of bytes from:
  • an iterable yielding integers in range(256)

  • a text string encoded using the specified encoding

  • any object implementing the buffer API.

  • an integer

oaknut.hfe.write_hfe(tracks, *, version=3, track_encoding=TrackEncoding.ISOIBM_FM, bitrate_kbps=250, rpm=300, interface_mode=7)

An HFE file holding tracks.

tracks is indexed [track][side]; each entry is a sample string of "0", "1" and, in v3, "?" for weak samples, which must fill whole, byte-aligned groups of eight. Sides are padded with zero samples to whole bytes and to a common length.

Raises:

ValueError – The sides disagree in number, weak samples are given for v1 or misaligned, or (v3) a byte of samples would read back as an opcode.

Parameters:
Return type:

bytes

Decoding tracks

Decode FM and MFM tracks into sectors.

Every track is searched for both FM and MFM address marks at every bit offset, whatever the HFE header claims. That finds mixed-density tracks, and marks that no byte or word grid would line up with. Each ID address mark found becomes a DecodedSector, including ones with bad CRCs, duplicate or out-of-range IDs, odd sizes, or no data field. Nothing is discarded at this level; deciding which sectors make up the logical disc is the job of oaknut.hfe.flat.

An FM half-cell spans two samples at the usual 250 kbit/s HFE rate. The samples are combined into half-cells at each possible phase and every phase is decoded, so a transition that slips by one sample part-way round a track still decodes. Sectors found at more than one phase are merged, keeping the best read.

Sectors that span the index are decoded by reading on round the track.

class oaknut.hfe.Encoding(*values)

Recording density.

FM = 'FM'
MFM = 'MFM'
class oaknut.hfe.DataMark(*values)

The address mark that introduces a data field.

NORMAL = 'normal'
DELETED = 'deleted'
class oaknut.hfe.SectorID(cylinder, head, record, size_code)

The C/H/R/N fields of an ID address mark.

Parameters:
size(encoding)

Data-field length in bytes.

The 8271 (FM) honours three bits of the size code, up to 16 KiB; the WD1770 (and so any MFM disc) honours two, up to 1 KiB.

Parameters:

encoding (Encoding)

Return type:

int

class oaknut.hfe.DecodedSector(encoding, position, id, id_crc_ok, data_mark=None, data=None, data_crc_ok=None)

One ID address mark and the data field that follows it, if any.

Parameters:
position: int

Sample offset of the address mark from the start of the track.

data_mark: DataMark | None = None

None when no data mark follows the ID.

property readable: bool

Whether a controller would read this sector without error.

class oaknut.hfe.DecodedTrack(track, side, sectors, sample_count=0, weak_samples=0)

The sectors decoded from one side of one physical track.

Parameters:
sectors: tuple[DecodedSector, ...]

In order of position round the track.

class oaknut.hfe.DecodedDisc(header, tracks)

Every track of an HFE image, decoded.

Parameters:
tracks: tuple[DecodedTrack, ...]

Ordered by track, then side.

track(track, side)

The decoded track on side.

Raises:

KeyError – The image has no such track.

Parameters:
Return type:

DecodedTrack

oaknut.hfe.decode_track(samples, *, bitrate_kbps=250)

Every sector on one track’s samples, in order round the track.

Parameters:
Return type:

tuple[DecodedSector, …]

oaknut.hfe.decode_image(image)

Decode every track of image.

Parameters:

image (HFEImage)

Return type:

DecodedDisc

Flattening to a sector image

Flatten a decoded disc into a logical sector image.

The filesystems read sector images: each track’s sectors in record order, tracks in order, the sides of a double-sided disc interleaved track by track (the .dsd and .adl layout). Flattening chooses the disc’s standard sector layout from what was decoded and fills each slot of it from the best read of that sector.

The layout is inferred, not taken from the HFE header:

  • the dominant density and sector size among readable sectors;

  • the usual number of sectors per track, and the first record number;

  • whether the capture is double-stepped (a 40-track disc read in an 80-track drive, so that each logical track occupies two physical tracks), from whether the ID fields’ cylinder numbers run at half the physical track number;

  • the number of tracks and sides that hold any standard sector.

A slot is filled from a readable sector on the matching physical track whose ID has the slot’s cylinder, record and size. Sectors that claim another cylinder, a record outside the layout, or another size are left out; they are visible in the sector report. A slot with no readable sector is filled with fill bytes and listed in FlatImage.missing.

class oaknut.hfe.SectorLayout(encoding, sector_size, sectors_per_track, first_record, num_tracks, num_sides, double_stepped=False)

The standard sector layout inferred for a disc.

Parameters:
  • encoding (Encoding)

  • sector_size (int)

  • sectors_per_track (int)

  • first_record (int)

  • num_tracks (int)

  • num_sides (int)

  • double_stepped (bool)

class oaknut.hfe.FlatImage(data, layout, missing=(), suffix=None)

A logical sector image made from an HFE disc.

Parameters:
layout: SectorLayout | None

None when no standard sector could be decoded at all.

missing: tuple[tuple[int, int, int], ...] = ()

(track, side, record) of each slot no readable sector filled.

suffix: str | None = None

The conventional extension for an image with this layout, if any.

oaknut.hfe.flatten(disc, *, fill=0)

The logical sector image of disc, with unreadable slots fill-ed.

Parameters:
Return type:

FlatImage

oaknut.hfe.flatten_hfe(filepath, *, fill=0)

Decode and flatten the HFE file at filepath.

Parameters:
Return type:

FlatImage

The sector report

What makes each track and sector of a decoded disc non-standard.

Copy protection works by putting things on a disc that a normal format would not: IDs claiming another cylinder or head, records outside the usual range, odd sector sizes, duplicate IDs, deliberate CRC errors, deleted-data marks, missing data fields, weak bits, and tracks beyond the formatted range. analyse() compares every decoded track and sector with the disc’s inferred SectorLayout and names each departure from it, so a protection scheme can be read off the report.

class oaknut.hfe.TrackFinding(track, side, decoded, sectors, remarks)

One physical track side and how it departs from the layout.

Parameters:
property anomalous: bool

Whether the track or any of its sectors drew a remark.

class oaknut.hfe.SectorFinding(sector, remarks)

One decoded sector and how it departs from the layout.

Parameters:
oaknut.hfe.analyse(disc, layout)

Findings for every track of disc against layout.

With no layout (nothing standard could be decoded) only the layout-independent remarks are made.

Parameters:
Return type:

tuple[TrackFinding, …]

Building tracks

Build FM and MFM tracks from sector descriptions.

The inverse of oaknut.hfe.decode: lay sectors out as an Acorn controller would format them, then convert the half-cells to HFE samples for oaknut.hfe.container.write_hfe(). Each FormattedSector can carry the defects copy protection uses — any ID, a bad ID or data CRC, a deleted-data mark, a missing data field — so decoder behaviour can be tested without real captures of every trick.

Ported from Beebium’s TrackBuilder.

class oaknut.hfe.FormattedSector(id, data=b'', deleted=False, omit_data=False, bad_id_crc=False, bad_data_crc=False)

One sector as formatted onto a track.

Parameters:
data: bytes = b''

The data field; padded with zeros, or truncated, to the ID’s size.

omit_data: bool = False

Write no data field at all after the ID.

bad_id_crc: bool = False

Corrupt the stored ID CRC.

bad_data_crc: bool = False

Corrupt the stored data CRC.

oaknut.hfe.build_track(sectors, encoding, *, half_cells=None)

The half-cells of a track holding sectors in the order given.

The track is padded with gap bytes to half_cells when given (one revolution at 300 rpm is 50,000 FM or 100,000 MFM half-cells).

Raises:

ValueError – The sectors do not fit in half_cells.

Parameters:
Return type:

str

oaknut.hfe.track_samples(half_cells, encoding, bitrate_kbps=250)

HFE samples for half_cells, each transition at the end of its cell.

Parameters:
Return type:

str

oaknut.hfe.samples_per_half_cell(encoding, bitrate_kbps=250)

HFE samples per half-cell: an FM half-cell is 4 µs, an MFM one 2 µs.

Raises:

ValueError – The bit rate does not give a whole number of samples.

Parameters:
Return type:

int

oaknut.hfe.TRACK_SAMPLES_300RPM_250KBPS = 100000

int([x]) -> integer int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments are given. If x is a number, return x.__int__(). For floating-point numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string, bytes, or bytearray instance representing an integer literal in the given base. The literal can be preceded by ‘+’ or ‘-’ and be surrounded by whitespace. The base defaults to 10. Valid bases are 0 and 2-36. Base 0 means to interpret the base from the string as an integer iteral. >>> int(‘0b100’, base=0) 4

The decoder plug-in

class oaknut.hfe.HFEDecoder(name, **kwargs)

HFE (HxC Floppy Emulator) flux images, v1 and v3, read-only.

FM and MFM tracks are decoded to sectors and flattened into the disc’s standard sector layout; unreadable sectors read as zeros.

Parameters:

name (str)

recognises(prefix)

Whether an image starting with prefix is in this format.

Parameters:

prefix (bytes)

Return type:

bool

decode(data)

The sector image decoded from the whole image data.

Raises:

DataError – data is in this format but cannot be decoded.

Parameters:

data (bytes)

Return type:

DecodedImage

Exceptions

exception oaknut.hfe.HFEError(*args, exit_code=None)

Base exception for HFE errors.

Parameters:
  • args (object)

  • exit_code (Optional[ExitCode])

Return type:

None

exception oaknut.hfe.HFEFormatError(*args, exit_code=None)

The file is not a well-formed HFE v1 or v3 image.

Parameters:
  • args (object)

  • exit_code (Optional[ExitCode])

Return type:

None