dtourolle
c0b1e78f7c
rawler panics on some input rather than returning Err — a DNG whose IFD claims a >50000 px image, which the reference library has: a 521 MB stitched panorama, IMG_4181-Pano.dng. On a worker thread a panic is the end of the thread, so the face sweep that met it stopped thirteen seconds in, three sweeps running on the tablet and three on the desktop, with "17301 image(s) to index" as the last word. FR-RAW-4 says a malformed file must not abort a batch, and that is this crate's promise whatever the library beneath it does: every entry point that calls into rawler now runs under catch_unwind, and a file that panics the decoder is one failed file with the panic's message in the error. Verified on the panorama itself: metadata reads, decode returns the error, the thread survives. The crash hook still records the panic, which is right — it is a defect in a dependency and the record is how it gets reported.
413 lines
15 KiB
Rust
413 lines
15 KiB
Rust
//! Embedded preview extraction — the fast display path.
|
||
//!
|
||
//! Every RAW container carries one or more JPEG previews, often at or near
|
||
//! full resolution. Extracting one costs a fraction of a full decode, and is
|
||
//! what makes culling feel instant (FR-CULL-1, NFR-P13: 50 ms per image).
|
||
//!
|
||
//! It is also what makes remote browsing viable: fetching ~1-3 MB of preview
|
||
//! from an 80 MB file over WebDAV is the difference between usable and not on
|
||
//! mobile data (FR-NC-3).
|
||
|
||
use crate::DecodeError;
|
||
|
||
/// How much of a file header to read when locating a preview.
|
||
///
|
||
/// Enough to cover the IFD structure of the TIFF-derived formats. Sized for
|
||
/// remote range requests, where every byte costs.
|
||
pub const PREVIEW_PROBE_BYTES: u64 = 256 * 1024;
|
||
|
||
/// A decoded preview image, RGBA8.
|
||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||
pub struct Preview {
|
||
pub width: u32,
|
||
pub height: u32,
|
||
/// Tightly packed RGBA, 4 bytes per pixel.
|
||
pub rgba: Vec<u8>,
|
||
}
|
||
|
||
impl Preview {
|
||
/// TRACES: FR-DEV-3h
|
||
/// Turn the pixels the right way up, in place.
|
||
///
|
||
/// Every path that shows a preview without the GPU needs this: the grid's
|
||
/// thumbnails, and the read-only fallback develop shows when no decoder
|
||
/// could open the file. An embedded preview is written in the sensor's
|
||
/// orientation, not the photograph's, so a phone or a camera held sideways
|
||
/// fills the grid with frames on their side until this runs.
|
||
///
|
||
/// Done before [`Self::downscale_to`] would be wasteful and after it is
|
||
/// not: a quarter turn is a permutation, so it costs the same either way,
|
||
/// and doing it on the smaller buffer moves a fraction of the bytes.
|
||
///
|
||
/// The turn itself is [`dr_types::Orientation::into_shown`], which every
|
||
/// other consumer of an orientation in this codebase also goes through.
|
||
/// That is deliberate: a hand-written permutation per caller is how two of
|
||
/// them come to disagree, and a disagreement here shows as a thumbnail
|
||
/// facing the other way from the develop view.
|
||
pub fn apply_orientation(&mut self, orientation: dr_types::Orientation) {
|
||
let (rgba, dw, dh) = orientation.into_shown(&self.rgba, self.width, self.height, 4);
|
||
self.rgba = rgba;
|
||
self.width = dw;
|
||
self.height = dh;
|
||
}
|
||
|
||
/// Downscale in place to fit within `max_dim` on the long edge.
|
||
///
|
||
/// A 5472x3648 preview is 79.8 MB of RGBA — far more than a grid cell or
|
||
/// even a 4K viewport needs, and enough to exhaust a phone's budget after
|
||
/// a handful of images (NFR-RES-1). Box-filtered rather than nearest, so
|
||
/// downscaled thumbnails do not alias.
|
||
pub fn downscale_to(&mut self, max_dim: u32) {
|
||
let longest = self.width.max(self.height);
|
||
if longest <= max_dim || longest == 0 {
|
||
return;
|
||
}
|
||
let scale = max_dim as f32 / longest as f32;
|
||
let (nw, nh) = (
|
||
((self.width as f32 * scale).round() as u32).max(1),
|
||
((self.height as f32 * scale).round() as u32).max(1),
|
||
);
|
||
|
||
let mut out = vec![0u8; (nw as usize) * (nh as usize) * 4];
|
||
let x_ratio = self.width as f32 / nw as f32;
|
||
let y_ratio = self.height as f32 / nh as f32;
|
||
|
||
for y in 0..nh {
|
||
let y0 = (y as f32 * y_ratio) as u32;
|
||
let y1 = (((y + 1) as f32 * y_ratio) as u32)
|
||
.min(self.height)
|
||
.max(y0 + 1);
|
||
for x in 0..nw {
|
||
let x0 = (x as f32 * x_ratio) as u32;
|
||
let x1 = (((x + 1) as f32 * x_ratio) as u32)
|
||
.min(self.width)
|
||
.max(x0 + 1);
|
||
|
||
let (mut r, mut g, mut b, mut n) = (0u32, 0u32, 0u32, 0u32);
|
||
for sy in y0..y1 {
|
||
for sx in x0..x1 {
|
||
let i = ((sy * self.width + sx) * 4) as usize;
|
||
r += self.rgba[i] as u32;
|
||
g += self.rgba[i + 1] as u32;
|
||
b += self.rgba[i + 2] as u32;
|
||
n += 1;
|
||
}
|
||
}
|
||
let n = n.max(1);
|
||
let o = ((y * nw + x) * 4) as usize;
|
||
out[o] = (r / n) as u8;
|
||
out[o + 1] = (g / n) as u8;
|
||
out[o + 2] = (b / n) as u8;
|
||
out[o + 3] = 255;
|
||
}
|
||
}
|
||
|
||
self.rgba = out;
|
||
self.width = nw;
|
||
self.height = nh;
|
||
}
|
||
|
||
/// Whether this is large enough to be worth displaying at `target`.
|
||
///
|
||
/// Some bodies embed thumbnails only a few hundred pixels wide — Sony is
|
||
/// the documented case. Displaying one where a larger render is wanted
|
||
/// shows a soft image the user discovers only on zoom, so the caller
|
||
/// should background-render instead (M-11).
|
||
pub fn is_useful_at(&self, target: u32) -> bool {
|
||
self.width.max(self.height) >= target
|
||
}
|
||
}
|
||
|
||
/// TRACES: FR-CULL-1 | NFR-P13
|
||
/// Which embedded image to extract.
|
||
///
|
||
/// Containers carry several at different sizes, and decoding the
|
||
/// full-resolution one to fill a grid cell is pure waste.
|
||
///
|
||
/// **Measured caveat (rawler 0.7.2):** the CR2 decoder implements only
|
||
/// `full_image`; `thumbnail_image` and `preview_image` are unimplemented trait
|
||
/// defaults returning `None`. So on Canon CR2 every rung currently resolves to
|
||
/// the full-resolution JPEG at ~250 ms — 5× over NFR-P13's 50 ms budget.
|
||
///
|
||
/// Three ways out, in increasing cost: extract the smaller IFD ourselves
|
||
/// (CR2 carries a 160×120 thumbnail and a ~1620×1080 preview in IFD1/IFD2),
|
||
/// contribute the methods upstream, or cache a downscaled proxy on first
|
||
/// sight. The ladder is written now so that fixing it is a decoder change
|
||
/// rather than a change to every caller.
|
||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||
pub enum PreviewSize {
|
||
/// Smallest available. Grid cells and rapid culling.
|
||
Thumbnail,
|
||
/// Mid-sized where the container has one. Single-image view.
|
||
Screen,
|
||
/// Largest available, usually full sensor resolution. Only where the
|
||
/// display genuinely needs it.
|
||
Full,
|
||
}
|
||
|
||
/// TRACES: FR-CULL-2 | FR-NC-3 | M-10
|
||
/// Extract and decode an embedded preview at the requested size.
|
||
///
|
||
/// Takes bytes rather than a reader, because the caller usually has them
|
||
/// already: a range read locally, or a `Range:` request remotely. Forcing a
|
||
/// `Read + Seek` here would push remote callers into buffering the whole file.
|
||
///
|
||
/// Falls through the ladder — a container without the requested size yields
|
||
/// the next available rather than failing (FR-CULL-2).
|
||
///
|
||
/// Returns [`DecodeError::NoPreview`] where there is none at all: a
|
||
/// fall-through signal, not a failure (see [`DecodeError::has_fallback`]).
|
||
pub fn extract_preview(bytes: &[u8], size: PreviewSize) -> Result<Preview, DecodeError> {
|
||
crate::error::guarded("preview", || extract_preview_unguarded(bytes, size))
|
||
}
|
||
|
||
fn extract_preview_unguarded(bytes: &[u8], size: PreviewSize) -> Result<Preview, DecodeError> {
|
||
use rawler::rawsource::RawSource;
|
||
|
||
// A plain JPEG *is* its own preview — rawler has no decoder for one, and
|
||
// a mixed folder must display sensibly (M-9).
|
||
if bytes.starts_with(&[0xFF, 0xD8, 0xFF]) {
|
||
return decode_jpeg(bytes);
|
||
}
|
||
|
||
let source = RawSource::new_from_slice(bytes);
|
||
let decoder =
|
||
rawler::get_decoder(&source).map_err(|e| DecodeError::Unsupported(e.to_string()))?;
|
||
let params = Default::default();
|
||
|
||
// Preference order per requested size, each falling through to the next.
|
||
let attempts: &[PreviewSize] = match size {
|
||
PreviewSize::Thumbnail => &[
|
||
PreviewSize::Thumbnail,
|
||
PreviewSize::Screen,
|
||
PreviewSize::Full,
|
||
],
|
||
PreviewSize::Screen => &[
|
||
PreviewSize::Screen,
|
||
PreviewSize::Full,
|
||
PreviewSize::Thumbnail,
|
||
],
|
||
PreviewSize::Full => &[PreviewSize::Full, PreviewSize::Screen],
|
||
};
|
||
|
||
for attempt in attempts {
|
||
let got = match attempt {
|
||
PreviewSize::Thumbnail => decoder.thumbnail_image(&source, ¶ms),
|
||
PreviewSize::Screen => decoder.preview_image(&source, ¶ms),
|
||
PreviewSize::Full => decoder.full_image(&source, ¶ms),
|
||
};
|
||
if let Ok(Some(img)) = got {
|
||
let rgb = img.to_rgb8();
|
||
let (width, height) = (rgb.width(), rgb.height());
|
||
if width > 0 && height > 0 {
|
||
return Ok(Preview {
|
||
width,
|
||
height,
|
||
rgba: rgb_to_rgba(rgb.as_raw(), width, height),
|
||
});
|
||
}
|
||
}
|
||
}
|
||
|
||
Err(DecodeError::NoPreview)
|
||
}
|
||
|
||
/// Extract the largest available preview.
|
||
///
|
||
/// Convenience over [`extract_preview`]; prefer naming a size explicitly.
|
||
pub fn extract_embedded_preview(bytes: &[u8]) -> Result<Preview, DecodeError> {
|
||
extract_preview(bytes, PreviewSize::Full)
|
||
}
|
||
|
||
/// Decode a standalone JPEG (an embedded preview already sliced out, or a
|
||
/// JPEG file).
|
||
pub fn decode_jpeg(bytes: &[u8]) -> Result<Preview, DecodeError> {
|
||
let mut d = zune_jpeg::JpegDecoder::new(bytes);
|
||
let pixels = d
|
||
.decode()
|
||
.map_err(|e| DecodeError::CorruptPreview(e.to_string()))?;
|
||
let info = d
|
||
.info()
|
||
.ok_or_else(|| DecodeError::CorruptPreview("no image info".into()))?;
|
||
|
||
let (w, h) = (info.width as u32, info.height as u32);
|
||
let expected = (w as usize) * (h as usize);
|
||
|
||
// zune yields RGB or grayscale depending on the source; normalise both to
|
||
// RGBA so callers have one representation.
|
||
let rgba = match pixels.len() / expected.max(1) {
|
||
3 => rgb_to_rgba(&pixels, w, h),
|
||
1 => pixels.iter().flat_map(|&g| [g, g, g, 255]).collect(),
|
||
4 => pixels,
|
||
n => {
|
||
return Err(DecodeError::CorruptPreview(format!(
|
||
"unexpected {n} channels"
|
||
)))
|
||
}
|
||
};
|
||
|
||
Ok(Preview {
|
||
width: w,
|
||
height: h,
|
||
rgba,
|
||
})
|
||
}
|
||
|
||
fn rgb_to_rgba(rgb: &[u8], w: u32, h: u32) -> Vec<u8> {
|
||
let n = (w as usize) * (h as usize);
|
||
let mut out = Vec::with_capacity(n * 4);
|
||
for px in rgb.chunks_exact(3).take(n) {
|
||
out.extend_from_slice(&[px[0], px[1], px[2], 255]);
|
||
}
|
||
out
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
/// A preview whose every pixel encodes its own coordinates, so a
|
||
/// misplaced one is identifiable rather than merely wrong.
|
||
fn coded(width: u32, height: u32) -> Preview {
|
||
let mut rgba = Vec::with_capacity((width * height * 4) as usize);
|
||
for y in 0..height {
|
||
for x in 0..width {
|
||
rgba.extend_from_slice(&[x as u8, y as u8, 0, 255]);
|
||
}
|
||
}
|
||
Preview {
|
||
width,
|
||
height,
|
||
rgba,
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn a_quarter_turn_moves_every_pixel_where_the_orientation_says() {
|
||
// Tag 6: the stored image's first row becomes the displayed right
|
||
// edge, its first column the displayed top. A 4x2 landscape preview
|
||
// therefore comes out 2x4 portrait, with stored (0,0) at the top right.
|
||
let mut p = coded(4, 2);
|
||
p.apply_orientation(dr_types::Orientation::from_exif(6));
|
||
|
||
assert_eq!((p.width, p.height), (2, 4));
|
||
let at = |x: u32, y: u32| {
|
||
let i = ((y * p.width + x) * 4) as usize;
|
||
(p.rgba[i], p.rgba[i + 1])
|
||
};
|
||
// Displayed top-right reads stored (0, 0).
|
||
assert_eq!(at(1, 0), (0, 0));
|
||
// Displayed top-left reads stored (0, 1) — the last row of column 0.
|
||
assert_eq!(at(0, 0), (0, 1));
|
||
// Displayed bottom-right reads stored (3, 0).
|
||
assert_eq!(at(1, 3), (3, 0));
|
||
}
|
||
|
||
#[test]
|
||
fn an_upright_file_is_left_untouched() {
|
||
// The common case, and the one where an unnecessary reallocation
|
||
// would be paid on every thumbnail in the library.
|
||
let original = coded(4, 2);
|
||
let mut p = original.clone();
|
||
p.apply_orientation(dr_types::Orientation::NORMAL);
|
||
assert_eq!(p, original);
|
||
}
|
||
|
||
#[test]
|
||
fn every_orientation_preserves_the_pixels_it_was_given() {
|
||
// A turn or a mirror is a permutation: the same bytes, rearranged.
|
||
// Anything else means a pixel was dropped, duplicated or read out of
|
||
// bounds — and the bounds case would have panicked first.
|
||
for tag in 1..=8u16 {
|
||
let orientation = dr_types::Orientation::from_exif(tag);
|
||
let mut p = coded(5, 3);
|
||
p.apply_orientation(orientation);
|
||
|
||
assert_eq!(
|
||
(p.width, p.height),
|
||
orientation.oriented_size(5, 3),
|
||
"tag {tag}"
|
||
);
|
||
let mut got: Vec<_> = p.rgba.chunks(4).map(|c| (c[0], c[1])).collect();
|
||
got.sort_unstable();
|
||
let mut want: Vec<_> = coded(5, 3).rgba.chunks(4).map(|c| (c[0], c[1])).collect();
|
||
want.sort_unstable();
|
||
assert_eq!(got, want, "tag {tag}");
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn size_preference_falls_through_in_order() {
|
||
// A container missing the requested size must yield the next
|
||
// available rather than failing (FR-CULL-2).
|
||
// Ordering is asserted here; behaviour against real files is covered
|
||
// by the smoke example.
|
||
assert_ne!(PreviewSize::Thumbnail, PreviewSize::Full);
|
||
}
|
||
|
||
#[test]
|
||
fn usefulness_is_judged_on_the_long_edge() {
|
||
let p = Preview {
|
||
width: 1600,
|
||
height: 1067,
|
||
rgba: Vec::new(),
|
||
};
|
||
assert!(p.is_useful_at(1024));
|
||
assert!(p.is_useful_at(1600));
|
||
// A body embedding only a small thumbnail must trigger a background
|
||
// render rather than showing a soft image.
|
||
assert!(!p.is_useful_at(2048));
|
||
}
|
||
|
||
#[test]
|
||
fn downscale_preserves_aspect_and_bounds_memory() {
|
||
let mut p = Preview {
|
||
width: 5472,
|
||
height: 3648,
|
||
rgba: vec![128; 5472 * 3648 * 4],
|
||
};
|
||
assert_eq!(p.rgba.len(), 79_847_424);
|
||
|
||
p.downscale_to(2048);
|
||
assert_eq!(p.width, 2048);
|
||
assert_eq!(p.height, 1365, "aspect preserved");
|
||
assert_eq!(p.rgba.len(), (2048 * 1365 * 4) as usize);
|
||
// A flat source must stay flat through the box filter.
|
||
assert!(p
|
||
.rgba
|
||
.chunks_exact(4)
|
||
.all(|px| px[0] == 128 && px[3] == 255));
|
||
}
|
||
|
||
#[test]
|
||
fn downscale_is_a_noop_when_already_small() {
|
||
let mut p = Preview {
|
||
width: 720,
|
||
height: 480,
|
||
rgba: vec![7; 720 * 480 * 4],
|
||
};
|
||
let before = p.rgba.len();
|
||
p.downscale_to(2048);
|
||
assert_eq!((p.width, p.height, p.rgba.len()), (720, 480, before));
|
||
}
|
||
|
||
#[test]
|
||
fn rgb_expands_to_rgba_opaque() {
|
||
let rgb = [10, 20, 30, 40, 50, 60];
|
||
let rgba = rgb_to_rgba(&rgb, 2, 1);
|
||
assert_eq!(rgba, vec![10, 20, 30, 255, 40, 50, 60, 255]);
|
||
}
|
||
|
||
#[test]
|
||
fn corrupt_jpeg_is_an_error_not_a_panic() {
|
||
// Untrusted input arrives here (NFR-SEC-1); it must never panic.
|
||
let err = decode_jpeg(&[0xFF, 0xD8, 0x00, 0x01, 0x02]).unwrap_err();
|
||
assert!(matches!(err, DecodeError::CorruptPreview(_)));
|
||
}
|
||
|
||
#[test]
|
||
fn empty_input_is_an_error_not_a_panic() {
|
||
assert!(decode_jpeg(&[]).is_err());
|
||
}
|
||
}
|