Files
navidrome/server/serve_index.go
T
Deluan Quintão 54de0dbc52 feat(server): implement FTS5-based full-text search (#5079)
* build: add sqlite_fts5 build tag to enable FTS5 support

* feat: add SearchBackend config option (default: fts)

* feat: add buildFTS5Query for safe FTS5 query preprocessing

* feat: add FTS5 search backend with config toggle, refactor legacy search

- Add searchExprFunc type and getSearchExpr() for backend selection
- Rename fullTextExpr to legacySearchExpr
- Add ftsSearchExpr using FTS5 MATCH subquery
- Update fullTextFilter in sql_restful.go to use configured backend

* feat: add FTS5 migration with virtual tables, triggers, and search_participants

Creates FTS5 virtual tables for media_file, album, and artist with
unicode61 tokenizer and diacritic folding. Adds search_participants
column, populates from JSON, and sets up INSERT/UPDATE/DELETE triggers.

* feat: populate search_participants in PostMapArgs for FTS5 indexing

* test: add FTS5 search integration tests

* fix: exclude FTS5 virtual tables from e2e DB restore

The restoreDB function iterates all tables in sqlite_master and
runs DELETE + INSERT to reset state. FTS5 contentless virtual tables
cannot be directly deleted from. Since triggers handle FTS5 sync
automatically, simply skip tables matching *_fts and *_fts_* patterns.

* build: add compile-time guard for sqlite_fts5 build tag

Same pattern as netgo: compilation fails with a clear error if
the sqlite_fts5 build tag is missing.

* build: add sqlite_fts5 tag to reflex dev server config

* build: extract GO_BUILD_TAGS variable in Makefile to avoid duplication

* fix: strip leading * from FTS5 queries to prevent "unknown special query" error

* feat: auto-append prefix wildcard to FTS5 search tokens for broader matching

Every plain search token now gets a trailing * appended (e.g., "love" becomes
"love*"), so searching for "love" also matches "lovelace", "lovely", etc.
Quoted phrases are preserved as exact matches without wildcards. Results are
ordered alphabetically by name/title, so shorter exact matches naturally
appear first.

* fix: clarify comments about FTS5 operator neutralization

The comments said "strip" but the code lowercases operators to
neutralize them (FTS5 operators are case-sensitive). Updated comments
to accurately describe the behavior.

* fix: use fmt.Sprintf for FTS5 phrase placeholders

The previous encoding used rune('0'+index) which silently breaks with
10+ quoted phrases. Use fmt.Sprintf for arbitrary index support.

* fix: validate and normalize SearchBackend config option

Normalize the value to lowercase and fall back to "fts" with a log
warning for unrecognized values. This prevents silent misconfiguration
from typos like "FTS", "Legacy", or "fts5".

* refactor: improve documentation for build tags and FTS5 requirements

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: convert FTS5 query and search backend normalization tests to DescribeTable format

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: add sqlite_fts5 build tag to golangci configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add UISearchDebounceMs configuration option and update related components

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: fall back to legacy search when SearchFullString is enabled

FTS5 is token-based and cannot match substrings within words, so
getSearchExpr now returns legacySearchExpr when SearchFullString
is true, regardless of SearchBackend setting.

* fix: add sqlite_fts5 build tag to CI pipeline and Dockerfile

* fix: add WHEN clauses to FTS5 AFTER UPDATE triggers

Added WHEN clauses to the media_file_fts_au, album_fts_au, and
artist_fts_au triggers so they only fire when FTS-indexed columns
actually change. Previously, every row update (e.g., play count, rating,
starred status) triggered an unnecessary delete+insert cycle in the FTS
shadow tables. The WHEN clauses use IS NOT for NULL-safe comparison of
each indexed column, avoiding FTS index churn for non-indexed updates.

* feat: add SearchBackend configuration option to data and insights components

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: enhance input sanitization for FTS5 by stripping additional punctuation and special characters

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add search_normalized column for punctuated name search (R.E.M., AC/DC)

Add index-time normalization and query-time single-letter collapsing to
fix FTS5 search for punctuated names. A new search_normalized column
stores concatenated forms of punctuated words (e.g., "R.E.M." → "REM",
"AC/DC" → "ACDC") and is indexed in FTS5 tables. At query time, runs of
consecutive single letters (from dot-stripping) are collapsed into OR
expressions like ("R E M" OR REM*) to match both the original tokens and
the normalized form. This enables searching by "R.E.M.", "REM", "AC/DC",
"ACDC", "A-ha", or "Aha" and finding the correct results.

* refactor: simplify isSingleUnicodeLetter to avoid []rune allocation

Use utf8.DecodeRuneInString to check for a single Unicode letter
instead of converting the entire string to a []rune slice.

* feat: define ftsSearchColumns for flexible FTS5 search column inclusion

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update collapseSingleLetterRuns to return quoted phrases for abbreviations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: punctuated word handling to improve processing of artist/album names

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add CJK support for search queries with LIKE filters

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance FTS5 search by adding album version support and CJK handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: search configuration to use structured options

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance search functionality to support punctuation-only queries and update related tests

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
2026-02-21 17:52:42 -05:00

186 lines
6.2 KiB
Go

package server
import (
"encoding/json"
"html/template"
"io"
"io/fs"
"net/http"
"os"
"path"
"strings"
"time"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/mime"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/slice"
"github.com/navidrome/navidrome/utils/str"
)
func Index(ds model.DataStore, fs fs.FS) http.HandlerFunc {
return serveIndex(ds, fs, nil)
}
func IndexWithShare(ds model.DataStore, fs fs.FS, shareInfo *model.Share) http.HandlerFunc {
return serveIndex(ds, fs, shareInfo)
}
// Injects the config in the `index.html` template
func serveIndex(ds model.DataStore, fs fs.FS, shareInfo *model.Share) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
c, err := ds.User(r.Context()).CountAll()
firstTime := c == 0 && err == nil
t, err := getIndexTemplate(r, fs)
if err != nil {
http.NotFound(w, r)
return
}
appConfig := map[string]any{
"version": consts.Version,
"firstTime": firstTime,
"variousArtistsId": consts.VariousArtistsID,
"baseURL": str.SanitizeText(strings.TrimSuffix(conf.Server.BasePath, "/")),
"loginBackgroundURL": str.SanitizeText(conf.Server.UILoginBackgroundURL),
"welcomeMessage": str.SanitizeText(conf.Server.UIWelcomeMessage),
"maxSidebarPlaylists": conf.Server.MaxSidebarPlaylists,
"enableTranscodingConfig": conf.Server.EnableTranscodingConfig,
"enableDownloads": conf.Server.EnableDownloads,
"enableFavourites": conf.Server.EnableFavourites,
"enableStarRating": conf.Server.EnableStarRating,
"defaultTheme": conf.Server.DefaultTheme,
"defaultLanguage": conf.Server.DefaultLanguage,
"defaultUIVolume": conf.Server.DefaultUIVolume,
"uiSearchDebounceMs": conf.Server.UISearchDebounceMs,
"enableCoverAnimation": conf.Server.EnableCoverAnimation,
"enableNowPlaying": conf.Server.EnableNowPlaying,
"gaTrackingId": conf.Server.GATrackingID,
"losslessFormats": strings.ToUpper(strings.Join(mime.LosslessFormats, ",")),
"devActivityPanel": conf.Server.DevActivityPanel,
"enableUserEditing": conf.Server.EnableUserEditing,
"enableSharing": conf.Server.EnableSharing,
"shareURL": conf.Server.ShareURL,
"defaultDownloadableShare": conf.Server.DefaultDownloadableShare,
"devSidebarPlaylists": conf.Server.DevSidebarPlaylists,
"lastFMEnabled": conf.Server.LastFM.Enabled,
"devShowArtistPage": conf.Server.DevShowArtistPage,
"devUIShowConfig": conf.Server.DevUIShowConfig,
"devNewEventStream": conf.Server.DevNewEventStream,
"listenBrainzEnabled": conf.Server.ListenBrainz.Enabled,
"enableExternalServices": conf.Server.EnableExternalServices,
"enableReplayGain": conf.Server.EnableReplayGain,
"defaultDownsamplingFormat": conf.Server.DefaultDownsamplingFormat,
"separator": string(os.PathSeparator),
"enableInspect": conf.Server.Inspect.Enabled,
"pluginsEnabled": conf.Server.Plugins.Enabled,
}
if strings.HasPrefix(conf.Server.UILoginBackgroundURL, "/") {
appConfig["loginBackgroundURL"] = path.Join(conf.Server.BasePath, conf.Server.UILoginBackgroundURL)
}
auth := handleLoginFromHeaders(ds, r)
if auth != nil {
appConfig["auth"] = auth
}
appConfigJson, err := json.Marshal(appConfig)
if err != nil {
log.Error(r, "Error converting config to JSON", "config", appConfig, err)
} else {
log.Trace(r, "Injecting config in index.html", "config", string(appConfigJson))
}
log.Debug("UI configuration", "appConfig", appConfig)
version := consts.Version
if version != "dev" {
version = "v" + version
}
data := map[string]any{
"AppConfig": string(appConfigJson),
"Version": version,
}
addShareData(r, data, shareInfo)
w.Header().Set("Content-Type", "text/html")
err = t.Execute(w, data)
if err != nil {
log.Error(r, "Could not execute `index.html` template", err)
}
}
}
func getIndexTemplate(r *http.Request, fs fs.FS) (*template.Template, error) {
t := template.New("initial state")
indexHtml, err := fs.Open("index.html")
if err != nil {
log.Error(r, "Could not find `index.html` template", err)
return nil, err
}
indexStr, err := io.ReadAll(indexHtml)
if err != nil {
log.Error(r, "Could not read from `index.html`", err)
return nil, err
}
t, err = t.Parse(string(indexStr))
if err != nil {
log.Error(r, "Error parsing `index.html`", err)
return nil, err
}
return t, nil
}
type shareData struct {
ID string `json:"id"`
Description string `json:"description"`
Downloadable bool `json:"downloadable"`
Tracks []shareTrack `json:"tracks"`
}
type shareTrack struct {
ID string `json:"id,omitempty"`
Title string `json:"title,omitempty"`
Artist string `json:"artist,omitempty"`
Album string `json:"album,omitempty"`
UpdatedAt time.Time `json:"updatedAt"`
Duration float32 `json:"duration,omitempty"`
}
func addShareData(r *http.Request, data map[string]any, shareInfo *model.Share) {
ctx := r.Context()
if shareInfo == nil || shareInfo.ID == "" {
return
}
sd := shareData{
ID: shareInfo.ID,
Description: shareInfo.Description,
Downloadable: shareInfo.Downloadable,
}
sd.Tracks = slice.Map(shareInfo.Tracks, func(mf model.MediaFile) shareTrack {
return shareTrack{
ID: mf.ID,
Title: mf.Title,
Artist: mf.Artist,
Album: mf.Album,
Duration: mf.Duration,
UpdatedAt: mf.UpdatedAt,
}
})
shareInfoJson, err := json.Marshal(sd)
if err != nil {
log.Error(ctx, "Error converting shareInfo to JSON", "config", shareInfo, err)
} else {
log.Trace(ctx, "Injecting shareInfo in index.html", "config", string(shareInfoJson))
}
if shareInfo.Description != "" {
data["ShareDescription"] = shareInfo.Description
} else {
data["ShareDescription"] = shareInfo.Contents
}
data["ShareURL"] = shareInfo.URL
data["ShareImageURL"] = shareInfo.ImageURL
data["ShareInfo"] = string(shareInfoJson)
}