54de0dbc52
* build: add sqlite_fts5 build tag to enable FTS5 support
* feat: add SearchBackend config option (default: fts)
* feat: add buildFTS5Query for safe FTS5 query preprocessing
* feat: add FTS5 search backend with config toggle, refactor legacy search
- Add searchExprFunc type and getSearchExpr() for backend selection
- Rename fullTextExpr to legacySearchExpr
- Add ftsSearchExpr using FTS5 MATCH subquery
- Update fullTextFilter in sql_restful.go to use configured backend
* feat: add FTS5 migration with virtual tables, triggers, and search_participants
Creates FTS5 virtual tables for media_file, album, and artist with
unicode61 tokenizer and diacritic folding. Adds search_participants
column, populates from JSON, and sets up INSERT/UPDATE/DELETE triggers.
* feat: populate search_participants in PostMapArgs for FTS5 indexing
* test: add FTS5 search integration tests
* fix: exclude FTS5 virtual tables from e2e DB restore
The restoreDB function iterates all tables in sqlite_master and
runs DELETE + INSERT to reset state. FTS5 contentless virtual tables
cannot be directly deleted from. Since triggers handle FTS5 sync
automatically, simply skip tables matching *_fts and *_fts_* patterns.
* build: add compile-time guard for sqlite_fts5 build tag
Same pattern as netgo: compilation fails with a clear error if
the sqlite_fts5 build tag is missing.
* build: add sqlite_fts5 tag to reflex dev server config
* build: extract GO_BUILD_TAGS variable in Makefile to avoid duplication
* fix: strip leading * from FTS5 queries to prevent "unknown special query" error
* feat: auto-append prefix wildcard to FTS5 search tokens for broader matching
Every plain search token now gets a trailing * appended (e.g., "love" becomes
"love*"), so searching for "love" also matches "lovelace", "lovely", etc.
Quoted phrases are preserved as exact matches without wildcards. Results are
ordered alphabetically by name/title, so shorter exact matches naturally
appear first.
* fix: clarify comments about FTS5 operator neutralization
The comments said "strip" but the code lowercases operators to
neutralize them (FTS5 operators are case-sensitive). Updated comments
to accurately describe the behavior.
* fix: use fmt.Sprintf for FTS5 phrase placeholders
The previous encoding used rune('0'+index) which silently breaks with
10+ quoted phrases. Use fmt.Sprintf for arbitrary index support.
* fix: validate and normalize SearchBackend config option
Normalize the value to lowercase and fall back to "fts" with a log
warning for unrecognized values. This prevents silent misconfiguration
from typos like "FTS", "Legacy", or "fts5".
* refactor: improve documentation for build tags and FTS5 requirements
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor: convert FTS5 query and search backend normalization tests to DescribeTable format
Signed-off-by: Deluan <deluan@navidrome.org>
* fix: add sqlite_fts5 build tag to golangci configuration
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: add UISearchDebounceMs configuration option and update related components
Signed-off-by: Deluan <deluan@navidrome.org>
* fix: fall back to legacy search when SearchFullString is enabled
FTS5 is token-based and cannot match substrings within words, so
getSearchExpr now returns legacySearchExpr when SearchFullString
is true, regardless of SearchBackend setting.
* fix: add sqlite_fts5 build tag to CI pipeline and Dockerfile
* fix: add WHEN clauses to FTS5 AFTER UPDATE triggers
Added WHEN clauses to the media_file_fts_au, album_fts_au, and
artist_fts_au triggers so they only fire when FTS-indexed columns
actually change. Previously, every row update (e.g., play count, rating,
starred status) triggered an unnecessary delete+insert cycle in the FTS
shadow tables. The WHEN clauses use IS NOT for NULL-safe comparison of
each indexed column, avoiding FTS index churn for non-indexed updates.
* feat: add SearchBackend configuration option to data and insights components
Signed-off-by: Deluan <deluan@navidrome.org>
* fix: enhance input sanitization for FTS5 by stripping additional punctuation and special characters
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: add search_normalized column for punctuated name search (R.E.M., AC/DC)
Add index-time normalization and query-time single-letter collapsing to
fix FTS5 search for punctuated names. A new search_normalized column
stores concatenated forms of punctuated words (e.g., "R.E.M." → "REM",
"AC/DC" → "ACDC") and is indexed in FTS5 tables. At query time, runs of
consecutive single letters (from dot-stripping) are collapsed into OR
expressions like ("R E M" OR REM*) to match both the original tokens and
the normalized form. This enables searching by "R.E.M.", "REM", "AC/DC",
"ACDC", "A-ha", or "Aha" and finding the correct results.
* refactor: simplify isSingleUnicodeLetter to avoid []rune allocation
Use utf8.DecodeRuneInString to check for a single Unicode letter
instead of converting the entire string to a []rune slice.
* feat: define ftsSearchColumns for flexible FTS5 search column inclusion
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: update collapseSingleLetterRuns to return quoted phrases for abbreviations
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor: punctuated word handling to improve processing of artist/album names
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: add CJK support for search queries with LIKE filters
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: enhance FTS5 search by adding album version support and CJK handling
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor: search configuration to use structured options
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: enhance search functionality to support punctuation-only queries and update related tests
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
186 lines
6.2 KiB
Go
186 lines
6.2 KiB
Go
package server
|
|
|
|
import (
|
|
"encoding/json"
|
|
"html/template"
|
|
"io"
|
|
"io/fs"
|
|
"net/http"
|
|
"os"
|
|
"path"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/navidrome/navidrome/conf"
|
|
"github.com/navidrome/navidrome/conf/mime"
|
|
"github.com/navidrome/navidrome/consts"
|
|
"github.com/navidrome/navidrome/log"
|
|
"github.com/navidrome/navidrome/model"
|
|
"github.com/navidrome/navidrome/utils/slice"
|
|
"github.com/navidrome/navidrome/utils/str"
|
|
)
|
|
|
|
func Index(ds model.DataStore, fs fs.FS) http.HandlerFunc {
|
|
return serveIndex(ds, fs, nil)
|
|
}
|
|
|
|
func IndexWithShare(ds model.DataStore, fs fs.FS, shareInfo *model.Share) http.HandlerFunc {
|
|
return serveIndex(ds, fs, shareInfo)
|
|
}
|
|
|
|
// Injects the config in the `index.html` template
|
|
func serveIndex(ds model.DataStore, fs fs.FS, shareInfo *model.Share) http.HandlerFunc {
|
|
return func(w http.ResponseWriter, r *http.Request) {
|
|
c, err := ds.User(r.Context()).CountAll()
|
|
firstTime := c == 0 && err == nil
|
|
|
|
t, err := getIndexTemplate(r, fs)
|
|
if err != nil {
|
|
http.NotFound(w, r)
|
|
return
|
|
}
|
|
appConfig := map[string]any{
|
|
"version": consts.Version,
|
|
"firstTime": firstTime,
|
|
"variousArtistsId": consts.VariousArtistsID,
|
|
"baseURL": str.SanitizeText(strings.TrimSuffix(conf.Server.BasePath, "/")),
|
|
"loginBackgroundURL": str.SanitizeText(conf.Server.UILoginBackgroundURL),
|
|
"welcomeMessage": str.SanitizeText(conf.Server.UIWelcomeMessage),
|
|
"maxSidebarPlaylists": conf.Server.MaxSidebarPlaylists,
|
|
"enableTranscodingConfig": conf.Server.EnableTranscodingConfig,
|
|
"enableDownloads": conf.Server.EnableDownloads,
|
|
"enableFavourites": conf.Server.EnableFavourites,
|
|
"enableStarRating": conf.Server.EnableStarRating,
|
|
"defaultTheme": conf.Server.DefaultTheme,
|
|
"defaultLanguage": conf.Server.DefaultLanguage,
|
|
"defaultUIVolume": conf.Server.DefaultUIVolume,
|
|
"uiSearchDebounceMs": conf.Server.UISearchDebounceMs,
|
|
"enableCoverAnimation": conf.Server.EnableCoverAnimation,
|
|
"enableNowPlaying": conf.Server.EnableNowPlaying,
|
|
"gaTrackingId": conf.Server.GATrackingID,
|
|
"losslessFormats": strings.ToUpper(strings.Join(mime.LosslessFormats, ",")),
|
|
"devActivityPanel": conf.Server.DevActivityPanel,
|
|
"enableUserEditing": conf.Server.EnableUserEditing,
|
|
"enableSharing": conf.Server.EnableSharing,
|
|
"shareURL": conf.Server.ShareURL,
|
|
"defaultDownloadableShare": conf.Server.DefaultDownloadableShare,
|
|
"devSidebarPlaylists": conf.Server.DevSidebarPlaylists,
|
|
"lastFMEnabled": conf.Server.LastFM.Enabled,
|
|
"devShowArtistPage": conf.Server.DevShowArtistPage,
|
|
"devUIShowConfig": conf.Server.DevUIShowConfig,
|
|
"devNewEventStream": conf.Server.DevNewEventStream,
|
|
"listenBrainzEnabled": conf.Server.ListenBrainz.Enabled,
|
|
"enableExternalServices": conf.Server.EnableExternalServices,
|
|
"enableReplayGain": conf.Server.EnableReplayGain,
|
|
"defaultDownsamplingFormat": conf.Server.DefaultDownsamplingFormat,
|
|
"separator": string(os.PathSeparator),
|
|
"enableInspect": conf.Server.Inspect.Enabled,
|
|
"pluginsEnabled": conf.Server.Plugins.Enabled,
|
|
}
|
|
if strings.HasPrefix(conf.Server.UILoginBackgroundURL, "/") {
|
|
appConfig["loginBackgroundURL"] = path.Join(conf.Server.BasePath, conf.Server.UILoginBackgroundURL)
|
|
}
|
|
auth := handleLoginFromHeaders(ds, r)
|
|
if auth != nil {
|
|
appConfig["auth"] = auth
|
|
}
|
|
appConfigJson, err := json.Marshal(appConfig)
|
|
if err != nil {
|
|
log.Error(r, "Error converting config to JSON", "config", appConfig, err)
|
|
} else {
|
|
log.Trace(r, "Injecting config in index.html", "config", string(appConfigJson))
|
|
}
|
|
|
|
log.Debug("UI configuration", "appConfig", appConfig)
|
|
version := consts.Version
|
|
if version != "dev" {
|
|
version = "v" + version
|
|
}
|
|
data := map[string]any{
|
|
"AppConfig": string(appConfigJson),
|
|
"Version": version,
|
|
}
|
|
addShareData(r, data, shareInfo)
|
|
|
|
w.Header().Set("Content-Type", "text/html")
|
|
err = t.Execute(w, data)
|
|
if err != nil {
|
|
log.Error(r, "Could not execute `index.html` template", err)
|
|
}
|
|
}
|
|
}
|
|
|
|
func getIndexTemplate(r *http.Request, fs fs.FS) (*template.Template, error) {
|
|
t := template.New("initial state")
|
|
indexHtml, err := fs.Open("index.html")
|
|
if err != nil {
|
|
log.Error(r, "Could not find `index.html` template", err)
|
|
return nil, err
|
|
}
|
|
indexStr, err := io.ReadAll(indexHtml)
|
|
if err != nil {
|
|
log.Error(r, "Could not read from `index.html`", err)
|
|
return nil, err
|
|
}
|
|
t, err = t.Parse(string(indexStr))
|
|
if err != nil {
|
|
log.Error(r, "Error parsing `index.html`", err)
|
|
return nil, err
|
|
}
|
|
return t, nil
|
|
}
|
|
|
|
type shareData struct {
|
|
ID string `json:"id"`
|
|
Description string `json:"description"`
|
|
Downloadable bool `json:"downloadable"`
|
|
Tracks []shareTrack `json:"tracks"`
|
|
}
|
|
|
|
type shareTrack struct {
|
|
ID string `json:"id,omitempty"`
|
|
Title string `json:"title,omitempty"`
|
|
Artist string `json:"artist,omitempty"`
|
|
Album string `json:"album,omitempty"`
|
|
UpdatedAt time.Time `json:"updatedAt"`
|
|
Duration float32 `json:"duration,omitempty"`
|
|
}
|
|
|
|
func addShareData(r *http.Request, data map[string]any, shareInfo *model.Share) {
|
|
ctx := r.Context()
|
|
if shareInfo == nil || shareInfo.ID == "" {
|
|
return
|
|
}
|
|
sd := shareData{
|
|
ID: shareInfo.ID,
|
|
Description: shareInfo.Description,
|
|
Downloadable: shareInfo.Downloadable,
|
|
}
|
|
sd.Tracks = slice.Map(shareInfo.Tracks, func(mf model.MediaFile) shareTrack {
|
|
return shareTrack{
|
|
ID: mf.ID,
|
|
Title: mf.Title,
|
|
Artist: mf.Artist,
|
|
Album: mf.Album,
|
|
Duration: mf.Duration,
|
|
UpdatedAt: mf.UpdatedAt,
|
|
}
|
|
})
|
|
|
|
shareInfoJson, err := json.Marshal(sd)
|
|
if err != nil {
|
|
log.Error(ctx, "Error converting shareInfo to JSON", "config", shareInfo, err)
|
|
} else {
|
|
log.Trace(ctx, "Injecting shareInfo in index.html", "config", string(shareInfoJson))
|
|
}
|
|
|
|
if shareInfo.Description != "" {
|
|
data["ShareDescription"] = shareInfo.Description
|
|
} else {
|
|
data["ShareDescription"] = shareInfo.Contents
|
|
}
|
|
data["ShareURL"] = shareInfo.URL
|
|
data["ShareImageURL"] = shareInfo.ImageURL
|
|
data["ShareInfo"] = string(shareInfoJson)
|
|
}
|