feat(subsonic): implement OpenSubsonic Transcoding extension (#4990)

* feat(subsonic): implement transcode decision logic and codec handling for media files

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): update codec limitation structure and decision logic for improved clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(transcoding): update bitrate handling to use kilobits per second (kbps) across transcode decision logic

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): simplify container alias handling in matchesContainer function

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(transcoding): enforce POST method for GetTranscodeDecision and handle non-POST requests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(transcoding): add enums for protocol, comparison operators, limitations, and codec profiles in transcode decision logic

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): streamline limitation checks and applyLimitation logic for improved readability and maintainability

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): replace strings.EqualFold with direct comparison for protocol and limitation checks

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): rename token methods to CreateTranscodeParams and ParseTranscodeParams for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): enhance logging for transcode decision process and client info conversion

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): rename TranscodeDecision to Decider and update related methods for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): enhance transcoding config lookup logic for audio codecs

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): enhance transcoding options with sample rate support and improve command handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): add bit depth support for audio transcoding and enhance related logic

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): enhance AAC command handling and support for audio channels in streaming

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): streamline transcoding logic by consolidating stream parameter handling and enhancing alias mapping

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcoding): update default command handling and add codec support for transcoding

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: implement noopDecider for transcoding decision handling in tests

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: address review findings for OpenSubsonic transcoding PR

Fix multiple issues identified during code review of the transcoding
extension: add missing return after error in shared stream handler
preventing nil pointer panic, replace dead r.Body nil check with
MaxBytesReader size limit, distinguish not-found from other DB errors,
fix bpsToKbps integer truncation with rounding, add "pcm" to
isLosslessFormat for consistency with model.IsLossless(), add
sampleRate/bitDepth/channels to streaming log, fix outdated test
comment, and add tests for conversion functions and GetTranscodeStream
parameter passing.

* feat(transcoding): add sourceUpdatedAt to decision and validate transcode parameters

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: small issues

Updated mock AAC transcoding command to use the new default (ipod with
fragmented MP4) matching the migration, ensuring tests exercise the same
buildDynamicArgs code path as production. Improved archiver test mock to
match on the whole StreamRequest struct instead of decomposing fields,
making it resilient to future field additions. Added named constants for
JWT claim keys in the transcode token and wrapped ParseTranscodeParams
errors with ErrTokenInvalid for consistency. Documented the IsLossless
BitDepth fallback heuristic as temporary until Codec column is populated.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(transcoding): adapt transcode claims to struct-based auth.Claims

Updated transcode token handling to use the struct-based auth.Claims
introduced on master, replacing the previous map[string]any approach.
Extended auth.Claims with transcoding-specific fields (MediaID, DirectPlay,
UpdatedAt, Channels, SampleRate, BitDepth) and added float64 fallback in
ClaimsFromToken for numeric claims that lose their Go type during JWT
string serialization. Also added the missing lyrics parameter to all
subsonic.New() calls in test files.

* feat(model): add ProbeData field and UpdateProbeData repository method

Add probe_data TEXT column to media_file for caching ffprobe results.
Add UpdateProbeData to MediaFileRepository interface and implementations.
Use hash:"ignore" tag so probe data doesn't affect MediaFile fingerprints.

* feat(ffmpeg): add ProbeAudioStream for authoritative audio metadata

Add ProbeAudioStream to FFmpeg interface, using ffprobe to extract
codec, profile, bitrate, sample rate, bit depth, and channels.
Parse bits_per_raw_sample as fallback for FLAC/ALAC bit depth.
Normalize "unknown" profile to empty string.
All parseProbeOutput tests use real ffprobe JSON from actual files.

* feat(transcoding): integrate ffprobe into transcode decisions

Add ensureProbed to probe media files on first transcode decision,
caching results in probe_data. Build SourceStream from probe data
with fallback to tag-based metadata.

Refactor decision logic to pass StreamDetails instead of MediaFile,
enabling codec profile limitations (e.g., audioProfile) to use
probe data. Add normalizeProbeCodec to map ffprobe codec names
(dsd_lsbf_planar, pcm_s16le) to internal names (dsd, pcm).

NewDecider now accepts ffmpeg.FFmpeg; wire_gen.go regenerated.

* feat(transcoding): add DevEnableMediaFileProbe config flag

Add DevEnableMediaFileProbe (default true) to allow disabling ffprobe-
based media file probing as a safety fallback. When disabled, the
decider uses tag-based metadata from the scanner instead.

* test(transcode): add ensureProbed unit tests

Test probing when ProbeData is empty, skipping when already set,
error propagation from ffprobe, and DevEnableMediaFileProbe flag.

* refactor(ffmpeg): use command constant and select_streams for ProbeAudioStream

Move ffprobe arguments to a probeAudioStreamCmd constant, following the
same pattern as extractImageCmd and probeCmd. Add -select_streams a:0 to
only probe the first audio stream, avoiding unnecessary parsing of video
and artwork streams. Derive the ffprobe binary path safely using
filepath.Dir/Base instead of replacing within the full path string.

* refactor(transcode): decouple transcode token claims from auth.Claims

Remove six transcode-specific fields (MediaID, DirectPlay, UpdatedAt,
Channels, SampleRate, BitDepth) from auth.Claims, which is shared with
session and share tokens. Transcode tokens are signed parameter-passing
tokens, not authentication tokens, so coupling them to auth created
misleading dependencies.

The transcode package now owns its own JWT claim serialization via
Decision.toClaimsMap() and paramsFromToken(), using generic
auth.EncodeToken/DecodeAndVerifyToken wrappers that keep TokenAuth
encapsulated. Wire format (JWT claim keys) is unchanged, so in-flight
tokens remain compatible.

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcode): simplify code after review

Extract getIntClaim helper to eliminate repeated int/int64/float64 JWT
claim extraction pattern in paramsFromToken and ClaimsFromToken. Rewrite
checkIntLimitation as a one-liner delegating to applyIntLimitation.
Return probe result from ensureProbed to avoid redundant JSON round-trip.
Extract toResponseStreamDetails helper and mediaTypeSong constant in
the API layer, and use transcode.ProtocolHTTP constant instead of
hardcoded string.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ffmpeg): enhance bit_rate parsing logic for audio streams

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(transcode): improve code review findings across transcode implementation

- Fix parseProbeData to return nil on JSON unmarshal failure instead of
  a zero-valued struct, preventing silent degradation of source stream details
- Use probe-resolved codec for lossless detection in buildSourceStream
  instead of the potentially stale scanner data
- Remove MediaFile.IsLossless() (dead code) and consolidate lossless
  detection in isLosslessFormat(), using codec name only — bit depth is
  not reliable since lossy codecs like ADPCM report non-zero values
- Add "wavpack" to lossless codec list (ffprobe codec_name for WavPack)
- Guard bpsToKbps against negative input values
- Fix misleading comment in buildTemplateArgs about conditional injection
- Avoid leaking internal error details in Subsonic API responses
- Add missing test for ErrNotFound branch in GetTranscodeDecision
- Add TODO for hardcoded protocol in toResponseStreamDetails

* refactor(transcode): streamline transcoding command lookup and format resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(transcode): implement server-side transcoding override for player formats

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(transcode): honor bit depth and channel constraints in transcoding selection

selectTranscodingOptions only checked sample rate when deciding whether
same-format transcoding was needed, ignoring requested bit depth and
channel reductions. This caused the streamer to return raw audio when
the transcode decision requested downmix or bit-depth conversion.

* refactor(transcode): unify streaming decision engine via MakeDecision

Move transcoding decision-making out of mediaStreamer and into the
subsonic Stream/Download handlers, using transcode.Decider.MakeDecision
as the single decision engine. This eliminates selectTranscodingOptions
and the mismatch between decision and streaming code paths (decision
used LookupTranscodeCommand with built-in fallbacks, while streaming
used FindByFormat which only checked the DB).

- Add DecisionOptions with SkipProbe to MakeDecision so the legacy
  streaming path never calls ffprobe
- Add buildLegacyClientInfo to translate legacy stream params (format,
  maxBitRate, DefaultDownsamplingFormat) into a synthetic ClientInfo
- Add resolveStreamRequest on the subsonic Router to resolve legacy
  params into a fully specified StreamRequest via MakeDecision
- Simplify DoStream to a dumb executor that receives pre-resolved params
- Remove selectTranscodingOptions entirely

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(transcode): move MediaStreamer into core/transcode and unify StreamRequest

Moved MediaStreamer, Stream, TranscodingCache and related types from
core/media_streamer.go into core/transcode/, eliminating the duplicate
StreamRequest type. The transcode.StreamRequest now carries all fields
(ID, Format, BitRate, SampleRate, BitDepth, Channels, Offset) and
ResolveStream returns a fully-populated value, removing manual field
copying at every call site. Also moved buildLegacyClientInfo into the
transcode package alongside ResolveStream, and unexported
ParseTranscodeParams since it was only used internally by
ValidateTranscodeParams.

* refactor(transcode): rename Decider methods and unexport Params type

Rename ResolveStream → ResolveRequest and ValidateTranscodeParams →
ResolveRequestFromToken for clarity and consistency. The new
ResolveRequestFromToken returns a StreamRequest directly (instead of
the intermediate Params type), eliminating manual Params→StreamRequest
conversion in callers. Unexport Params to params since it is now only
used internally for JWT token parsing.

* test(transcode): remove redundant tests and use constants

Remove tests that duplicate coverage from integration-level tests
(toClaimsMap, paramsFromToken round-trips, applyServerOverride direct
call, duplicate 410 handler test). Replace raw "http" strings with
ProtocolHTTP constant. Consolidate lossy -sample_fmt tests into
DescribeTable.

* refactor(transcode): split oversized files into focused modules

Split transcode.go and transcode_test.go into focused files by concern:
- decider.go: decision engine (MakeDecision, direct play/transcode evaluation, probe)
- token.go: JWT token encode/decode (params, toClaimsMap, paramsFromToken, CreateTranscodeParams, ResolveRequestFromToken)
- legacy_client.go: legacy Subsonic bridge (buildLegacyClientInfo, ResolveRequest)
- codec_test.go: isLosslessFormat and normalizeProbeCodec tests
- token_test.go: token round-trip and ResolveRequestFromToken tests

Moved the Decider interface from types.go to decider.go to keep it near
its implementation, and cleaned up types.go to contain only pure type
definitions and constants. No public API changes.

* refactor(transcode): reorder parameters in applyServerOverride function

Signed-off-by: Deluan <deluan@navidrome.org>

* test(e2e): add NewTestStream function and implement spyStreamer for testing

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
This commit is contained in:
Deluan Quintão
2026-03-08 23:57:49 -04:00
committed by GitHub
parent e1b3412999
commit ae1e0ddb11
51 changed files with 4828 additions and 352 deletions
+268 -8
View File
@@ -2,23 +2,49 @@ package ffmpeg
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"sync"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
)
// TranscodeOptions contains all parameters for a transcoding operation.
type TranscodeOptions struct {
Command string // DB command template (used to detect custom vs default)
Format string // Target format (mp3, opus, aac, flac)
FilePath string
BitRate int // kbps, 0 = codec default
SampleRate int // 0 = no constraint
Channels int // 0 = no constraint
BitDepth int // 0 = no constraint; valid values: 16, 24, 32
Offset int // seconds
}
// AudioProbeResult contains authoritative audio stream properties from ffprobe.
type AudioProbeResult struct {
Codec string `json:"codec"`
Profile string `json:"profile,omitempty"`
BitRate int `json:"bitRate"`
SampleRate int `json:"sampleRate"`
BitDepth int `json:"bitDepth"`
Channels int `json:"channels"`
}
type FFmpeg interface {
Transcode(ctx context.Context, command, path string, maxBitRate, offset int) (io.ReadCloser, error)
Transcode(ctx context.Context, opts TranscodeOptions) (io.ReadCloser, error)
ExtractImage(ctx context.Context, path string) (io.ReadCloser, error)
Probe(ctx context.Context, files []string) (string, error)
ProbeAudioStream(ctx context.Context, filePath string) (*AudioProbeResult, error)
CmdPath() (string, error)
IsAvailable() bool
Version() string
@@ -29,21 +55,26 @@ func New() FFmpeg {
}
const (
extractImageCmd = "ffmpeg -i %s -map 0:v -map -0:V -vcodec copy -f image2pipe -"
probeCmd = "ffmpeg %s -f ffmetadata"
extractImageCmd = "ffmpeg -i %s -map 0:v -map -0:V -vcodec copy -f image2pipe -"
probeCmd = "ffmpeg %s -f ffmetadata"
probeAudioStreamCmd = "ffprobe -v quiet -select_streams a:0 -print_format json -show_streams -show_format %s"
)
type ffmpeg struct{}
func (e *ffmpeg) Transcode(ctx context.Context, command, path string, maxBitRate, offset int) (io.ReadCloser, error) {
func (e *ffmpeg) Transcode(ctx context.Context, opts TranscodeOptions) (io.ReadCloser, error) {
if _, err := ffmpegCmd(); err != nil {
return nil, err
}
// First make sure the file exists
if err := fileExists(path); err != nil {
if err := fileExists(opts.FilePath); err != nil {
return nil, err
}
args := createFFmpegCommand(command, path, maxBitRate, offset)
var args []string
if isDefaultCommand(opts.Format, opts.Command) {
args = buildDynamicArgs(opts)
} else {
args = buildTemplateArgs(opts)
}
return e.start(ctx, args)
}
@@ -51,7 +82,6 @@ func (e *ffmpeg) ExtractImage(ctx context.Context, path string) (io.ReadCloser,
if _, err := ffmpegCmd(); err != nil {
return nil, err
}
// First make sure the file exists
if err := fileExists(path); err != nil {
return nil, err
}
@@ -81,6 +111,91 @@ func (e *ffmpeg) Probe(ctx context.Context, files []string) (string, error) {
return string(output), nil
}
func (e *ffmpeg) ProbeAudioStream(ctx context.Context, filePath string) (*AudioProbeResult, error) {
if _, err := ffmpegCmd(); err != nil {
return nil, err
}
if err := fileExists(filePath); err != nil {
return nil, err
}
args := createFFmpegCommand(probeAudioStreamCmd, filePath, 0, 0)
log.Trace(ctx, "Executing ffprobe command", "args", args)
cmd := exec.CommandContext(ctx, args[0], args[1:]...) // #nosec
output, err := cmd.Output()
if err != nil {
return nil, fmt.Errorf("running ffprobe on %q: %w", filePath, err)
}
return parseProbeOutput(output)
}
type probeOutput struct {
Streams []probeStream `json:"streams"`
Format probeFormat `json:"format"`
}
type probeFormat struct {
BitRate string `json:"bit_rate"`
}
type probeStream struct {
CodecName string `json:"codec_name"`
CodecType string `json:"codec_type"`
Profile string `json:"profile"`
SampleRate string `json:"sample_rate"`
BitRate string `json:"bit_rate"`
Channels int `json:"channels"`
BitsPerSample int `json:"bits_per_sample"`
BitsPerRawSample string `json:"bits_per_raw_sample"`
}
func parseProbeOutput(data []byte) (*AudioProbeResult, error) {
var output probeOutput
if err := json.Unmarshal(data, &output); err != nil {
return nil, fmt.Errorf("parsing ffprobe output: %w", err)
}
for _, s := range output.Streams {
if s.CodecType != "audio" {
continue
}
bitDepth := s.BitsPerSample
if bitDepth == 0 && s.BitsPerRawSample != "" {
bitDepth, _ = strconv.Atoi(s.BitsPerRawSample)
}
result := &AudioProbeResult{
Codec: s.CodecName,
Channels: s.Channels,
BitDepth: bitDepth,
}
// Profile: "unknown" → empty
if s.Profile != "" && !strings.EqualFold(s.Profile, "unknown") {
result.Profile = s.Profile
}
// Sample rate: string → int
if s.SampleRate != "" {
result.SampleRate, _ = strconv.Atoi(s.SampleRate)
}
// Bit rate: bps string → kbps int
if s.BitRate != "" {
bps, _ := strconv.Atoi(s.BitRate)
result.BitRate = bps / 1000
}
// Fallback to format-level bit_rate (needed for FLAC, Opus, etc.)
if result.BitRate == 0 && output.Format.BitRate != "" {
bps, _ := strconv.Atoi(output.Format.BitRate)
result.BitRate = bps / 1000
}
return result, nil
}
return nil, fmt.Errorf("no audio stream found in ffprobe output")
}
func (e *ffmpeg) CmdPath() (string, error) {
return ffmpegCmd()
}
@@ -156,6 +271,141 @@ func (j *ffCmd) wait() {
_ = j.out.Close()
}
// formatCodecMap maps target format to ffmpeg codec flag.
var formatCodecMap = map[string]string{
"mp3": "libmp3lame",
"opus": "libopus",
"aac": "aac",
"flac": "flac",
}
// formatOutputMap maps target format to ffmpeg output format flag (-f).
var formatOutputMap = map[string]string{
"mp3": "mp3",
"opus": "opus",
"aac": "ipod",
"flac": "flac",
}
// defaultCommands is used to detect whether a user has customized their transcoding command.
var defaultCommands = func() map[string]string {
m := make(map[string]string, len(consts.DefaultTranscodings))
for _, t := range consts.DefaultTranscodings {
m[t.TargetFormat] = t.Command
}
return m
}()
// isDefaultCommand returns true if the command matches the known default for this format.
func isDefaultCommand(format, command string) bool {
return defaultCommands[format] == command
}
// buildDynamicArgs programmatically constructs ffmpeg arguments for known formats,
// including all transcoding parameters (bitrate, sample rate, channels).
func buildDynamicArgs(opts TranscodeOptions) []string {
cmdPath, _ := ffmpegCmd()
args := []string{cmdPath, "-i", opts.FilePath}
if opts.Offset > 0 {
args = append(args, "-ss", strconv.Itoa(opts.Offset))
}
args = append(args, "-map", "0:a:0")
if codec, ok := formatCodecMap[opts.Format]; ok {
args = append(args, "-c:a", codec)
}
if opts.BitRate > 0 {
args = append(args, "-b:a", strconv.Itoa(opts.BitRate)+"k")
}
if opts.SampleRate > 0 {
args = append(args, "-ar", strconv.Itoa(opts.SampleRate))
}
if opts.Channels > 0 {
args = append(args, "-ac", strconv.Itoa(opts.Channels))
}
// Only pass -sample_fmt for lossless output formats where bit depth matters.
// Lossy codecs (mp3, aac, opus) handle sample format conversion internally,
// and passing interleaved formats like "s16" causes silent failures.
if opts.BitDepth >= 16 && isLosslessOutputFormat(opts.Format) {
args = append(args, "-sample_fmt", bitDepthToSampleFmt(opts.BitDepth))
}
args = append(args, "-v", "0")
if outputFmt, ok := formatOutputMap[opts.Format]; ok {
args = append(args, "-f", outputFmt)
}
// For AAC in MP4 container, enable fragmented MP4 for pipe-safe streaming
if opts.Format == "aac" {
args = append(args, "-movflags", "frag_keyframe+empty_moov")
}
args = append(args, "-")
return args
}
// buildTemplateArgs handles user-customized command templates, with dynamic injection
// of sample rate, channels, and bit depth when requested by the transcode decision.
// Note: these flags are injected unconditionally when non-zero, even if the template
// already includes them. FFmpeg uses the last occurrence of duplicate flags.
func buildTemplateArgs(opts TranscodeOptions) []string {
args := createFFmpegCommand(opts.Command, opts.FilePath, opts.BitRate, opts.Offset)
// Dynamically inject -ar, -ac, and -sample_fmt before the output target
if opts.SampleRate > 0 {
args = injectBeforeOutput(args, "-ar", strconv.Itoa(opts.SampleRate))
}
if opts.Channels > 0 {
args = injectBeforeOutput(args, "-ac", strconv.Itoa(opts.Channels))
}
if opts.BitDepth >= 16 && isLosslessOutputFormat(opts.Format) {
args = injectBeforeOutput(args, "-sample_fmt", bitDepthToSampleFmt(opts.BitDepth))
}
return args
}
// injectBeforeOutput inserts a flag and value before the trailing "-" (stdout output).
func injectBeforeOutput(args []string, flag, value string) []string {
if len(args) > 0 && args[len(args)-1] == "-" {
result := make([]string, 0, len(args)+2)
result = append(result, args[:len(args)-1]...)
result = append(result, flag, value, "-")
return result
}
return append(args, flag, value)
}
// isLosslessOutputFormat returns true if the format is a lossless audio format
// where preserving bit depth via -sample_fmt is meaningful.
// Note: this covers only formats ffmpeg can produce as output. For the full set of
// lossless formats used in transcoding decisions, see core/transcode/codec.go:isLosslessFormat.
func isLosslessOutputFormat(format string) bool {
switch strings.ToLower(format) {
case "flac", "alac", "wav", "aiff":
return true
}
return false
}
// bitDepthToSampleFmt converts a bit depth value to the ffmpeg sample_fmt string.
// FLAC only supports s16 and s32; for 24-bit sources, s32 is the correct format
// (ffmpeg packs 24-bit samples into 32-bit containers).
func bitDepthToSampleFmt(bitDepth int) string {
switch bitDepth {
case 16:
return "s16"
case 32:
return "s32"
default:
// 24-bit and other depths: use s32 (the next valid container size)
return "s32"
}
}
// Path will always be an absolute path
func createFFmpegCommand(cmd, path string, maxBitRate, offset int) []string {
var args []string
@@ -196,10 +446,20 @@ func fixCmd(cmd string) []string {
if s == "ffmpeg" || s == "ffmpeg.exe" {
split[i] = cmdPath
}
if s == "ffprobe" || s == "ffprobe.exe" {
split[i] = ffprobePath(cmdPath)
}
}
return split
}
// ffprobePath derives the ffprobe binary path from the resolved ffmpeg path.
func ffprobePath(ffmpegCmd string) string {
dir := filepath.Dir(ffmpegCmd)
base := filepath.Base(ffmpegCmd)
return filepath.Join(dir, strings.Replace(base, "ffmpeg", "ffprobe", 1))
}
func ffmpegCmd() (string, error) {
ffOnce.Do(func() {
if conf.Server.FFmpegPath != "" {