aa9391199182bb28fa4ffa222ec2a5fa02b77ee9
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 MaximilianandGitHub
|
a704e86ac1 | refactor: run Go modernize (#5002) | ||
|
Deluan QuintãoGitHubellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
f1fc2cd9b9 |
feat(plugins): experimental support for plugins (#3998)
* feat(plugins): add minimal test agent plugin with API definitions Signed-off-by: Deluan <[email protected]> * feat: add plugin manager with auto-registration and unique agent names Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator. * test: add Ginkgo suite and test for plugin manager auto-registration Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support. * test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility. Signed-off-by: Deluan <[email protected]> * feat(plugins): implement persistent compilation cache for WASM agent plugins Signed-off-by: Deluan <[email protected]> * feat(plugins): implement instance pooling for wasmAgent to improve resource management Signed-off-by: Deluan <[email protected]> * feat(plugins): enhance logging for wasmAgent and plugin manager operations Signed-off-by: Deluan <[email protected]> * feat(plugins): implement HttpService for handling HTTP requests in WASM plugins Also add a sample Wikimedia plugin Signed-off-by: Deluan <[email protected]> * feat(plugins): standardize error handling in wasmAgent and MinimalAgent Signed-off-by: Deluan <[email protected]> * refactor: clean up wikimedia plugin code Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy. * refactor: use unified SPARQLResult struct and parser for SPARQL responses Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability. * feat(plugins): improve error handling in HTTP request processing Signed-off-by: Deluan <[email protected]> * fix: background plugin compilation, logging, and race safety Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass. * perf: implement true lazy loading for agents Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass. * fix: ensure wasm plugin instances are closed via runtime.AddCleanup Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup. * refactor: break down long functions in plugin manager and agent Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go. Signed-off-by: Deluan <[email protected]> * refactor(plugins): rename wasmAgent to wasmArtistAgent Signed-off-by: Deluan <[email protected]> * feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests Signed-off-by: Deluan <[email protected]> * refactor(plugin): rename MinimalAgent for artist metadata service Signed-off-by: Deluan <[email protected]> * feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods Signed-off-by: Deluan <[email protected]> * refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin Signed-off-by: Deluan <[email protected]> * feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager Signed-off-by: Deluan <[email protected]> * feat(plugins): enhance plugin pool creation with custom runtime and precompilation support Signed-off-by: Deluan <[email protected]> * refactor(plugins): implement generic plugin pool and agent factory for improved service handling Signed-off-by: Deluan <[email protected]> * refactor(plugins): reorganize plugin management Signed-off-by: Deluan <[email protected]> * refactor(plugins): improve function signatures for clarity and consistency Signed-off-by: Deluan <[email protected]> * feat(plugins): implement background precompilation for plugins and agent factory creation Signed-off-by: Deluan <[email protected]> * refactor(plugins): include instanceID in logging for better traceability Signed-off-by: Deluan <[email protected]> * test(plugins): add tests for plugin pre-compilation and agent factory synchronization Signed-off-by: Deluan <[email protected]> * feat(plugins): add minimal album test agent plugin for AlbumMetadataService Signed-off-by: Deluan <[email protected]> * feat(plugins): rename fake artist and album test agent plugins for metadata services Signed-off-by: Deluan <[email protected]> * feat(makefile): add Makefile for building plugin WASM binaries Signed-off-by: Deluan <[email protected]> * feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services Signed-off-by: Deluan <[email protected]> * refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods Signed-off-by: Deluan <[email protected]> * refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized. * feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images Signed-off-by: Deluan <[email protected]> * refactor: remove wasm module pooling it was causing issues with the GC and the Close methods Signed-off-by: Deluan <[email protected]> * refactor: rename metadata service files to adapter naming convention Signed-off-by: Deluan <[email protected]> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <[email protected]> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <[email protected]> * fix: handle nil values in data redaction process Signed-off-by: Deluan <[email protected]> * fix: add timeout for plugin compilation to prevent indefinite blocking Signed-off-by: Deluan <[email protected]> * feat: implement ScrobblerService plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <[email protected]> * refactor: simplify generalization Signed-off-by: Deluan <[email protected]> * fix: tests Signed-off-by: Deluan <[email protected]> * refactor: enhance plugin management by improving scanning and loading mechanisms Signed-off-by: Deluan <[email protected]> * refactor: update plugin creation functions to return specific interfaces for better type safety Signed-off-by: Deluan <[email protected]> * refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety Signed-off-by: Deluan <[email protected]> * refactor: implement MediaMetadataService with combined artist and album methods Signed-off-by: Deluan <[email protected]> * refactor: improve MediaMetadataService plugin implementation and testing structure Signed-off-by: Deluan <[email protected]> * refactor: add tests for Adapter Media Agent and improve plugin documentation Signed-off-by: Deluan <[email protected]> * docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines Signed-off-by: Deluan <[email protected]> * refactor: enhance agent management with plugin loading and caching Signed-off-by: Deluan <[email protected]> * refactor: update agent discovery logic to include only local agent when no config is specified Signed-off-by: Deluan <[email protected]> * refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met. Signed-off-by: Deluan <[email protected]> * fix: correct file extension filter in goimports command Signed-off-by: Deluan <[email protected]> * refactor: use defer to unlock the mutex Signed-off-by: Deluan <[email protected]> * chore: move Cover Art Archive AlbumMetadataService plugins to an example folder Signed-off-by: Deluan <[email protected]> * fix: handle errors when creating media metadata and scrobbler service plugins Signed-off-by: Deluan <[email protected]> * fix: increase compilation timeout to one minute Signed-off-by: Deluan <[email protected]> * feat: add configurable plugin compilation timeout Signed-off-by: Deluan <[email protected]> * feat: implement plugin scrobbler support in PlayTracker Signed-off-by: Deluan <[email protected]> * feat: add context management and Stop method to buffered scrobbler Signed-off-by: Deluan <[email protected]> * feat: add username field to scrobbler requests and update logging Signed-off-by: Deluan <[email protected]> * fix: data race in test Signed-off-by: Deluan <[email protected]> * refactor: rename http proto files to host and update references Signed-off-by: Deluan <[email protected]> * refactor: remove unused plugin registration methods from manager Signed-off-by: Deluan <[email protected]> * feat: extend plugin manifests and implement plugin management commands Signed-off-by: Deluan <[email protected]> * Update utils/files.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip") Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * feat: add plugin dev workflow support Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely. Signed-off-by: Deluan <[email protected]> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <[email protected]> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <[email protected]> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <[email protected]> * feat(plugins): implement timer service with register and cancel functionality Signed-off-by: Deluan <[email protected]> * fix: lint errors Signed-off-by: Deluan <[email protected]> * feat(README): update documentation to include TimerCallbackService and its functionality Signed-off-by: Deluan <[email protected]> * feat(plugins): add InitService with OnInit method and initialization tracking - WIP Signed-off-by: Deluan <[email protected]> * feat(plugins): add tests for InitService and plugin initialization tracking Signed-off-by: Deluan <[email protected]> * feat(plugins): expand documentation on plugin system implementation and architecture Signed-off-by: Deluan <[email protected]> * fix: panic Signed-off-by: Deluan <[email protected]> * feat(plugins): redirect plugins' stderr to logs Signed-off-by: Deluan <[email protected]> * feat(plugins): add safe accessor methods for TimerService Signed-off-by: Deluan <[email protected]> * feat(plugins): add plugin-specific configuration support in InitRequest and documentation Signed-off-by: Deluan <[email protected]> * feat(plugins): add TimerCallbackService plugin adapter and integration Signed-off-by: Deluan <[email protected]> * refactor(plugins): rename services for consistency and clarity Signed-off-by: Deluan <[email protected]> * feat(plugins): add mutex for configuration access and clone plugin config Signed-off-by: Deluan <[email protected]> * refactor(tests): remove configtest dependency to prevent data races in integration tests Signed-off-by: Deluan <[email protected]> * refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type Signed-off-by: Deluan <[email protected]> * feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP Signed-off-by: Deluan <[email protected]> * feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size Signed-off-by: Deluan <[email protected]> * fix(plugins): correctly pass error to done function in wasmBasePlugin Signed-off-by: Deluan <[email protected]> * refactor(plugins): rename service types to capabilities for consistency Signed-off-by: Deluan <[email protected]> * refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure Signed-off-by: Deluan <[email protected]> * refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling Signed-off-by: Deluan <[email protected]> * refactor(plugins): rename InitService to LifecycleManagement for consistency Signed-off-by: Deluan <[email protected]> * refactor(plugins): fix instance ID logging in wasmBasePlugin Signed-off-by: Deluan <[email protected]> * refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error Signed-off-by: Deluan <[email protected]> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <[email protected]> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <[email protected]> * refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging Signed-off-by: Deluan <[email protected]> * feat(plugins): add config service for plugin-specific configuration management Signed-off-by: Deluan <[email protected]> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * feat(crontab): implement crontab service for scheduling and canceling jobs Signed-off-by: Deluan <[email protected]> * fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <[email protected]> (+1 squashed commit) Squashed commits: [325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <[email protected]> * feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService Signed-off-by: Deluan <[email protected]> * fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID Signed-off-by: Deluan <[email protected]> * refactor(scheduler): consolidate job scheduling logic into a single helper function Signed-off-by: Deluan <[email protected]> * refactor(plugin): rename GetInstance method to Instantiate for clarity Signed-off-by: Deluan <[email protected]> * feat(plugins): add WebSocket service for handling connections and messages Signed-off-by: Deluan <[email protected]> * feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking Signed-off-by: Deluan <[email protected]> * feat(websocket): enhance connection management and callback handling Signed-off-by: Deluan <[email protected]> * feat(manager): only create one adapter instance for each adapter/capability pair Signed-off-by: Deluan <[email protected]> * fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes Signed-off-by: Deluan <[email protected]> * fix: flaky test Signed-off-by: Deluan <[email protected]> * feat(plugin): refactor WebSocket service integration and improve error logging Signed-off-by: Deluan <[email protected]> * feat(plugin): add SchedulerCallback support and improve reconnection logic Signed-off-by: Deluan <[email protected]> * fix: test panic Signed-off-by: Deluan <[email protected]> * docs: add crypto-ticker plugin example to README Signed-off-by: Deluan <[email protected]> * feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration Signed-off-by: Deluan <[email protected]> * feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods Signed-off-by: Deluan <[email protected]> * feat(websocket): add error field to response messages for better error handling Signed-off-by: Deluan <[email protected]> * feat(cache): implement CacheService with string, int, float, and byte operations Signed-off-by: Deluan <[email protected]> * feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo Signed-off-by: Deluan <[email protected]> * refactor(cache): simplify cache service implementation and remove unnecessary synchronization Signed-off-by: Deluan <[email protected]> * feat(tests): add build step for test plugins in the test suite Signed-off-by: Deluan <[email protected]> * wip Signed-off-by: Deluan <[email protected]> * feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration Signed-off-by: Deluan <[email protected]> * feat(rpc): enhance activity image processing and improve error handling in Discord integration Signed-off-by: Deluan <[email protected]> * feat(discord): enhance activity state with artist list and add large text asset Signed-off-by: Deluan <[email protected]> * fix tests Signed-off-by: Deluan <[email protected]> * feat(artwork): implement ArtworkService for retrieving artwork URLs Signed-off-by: Deluan <[email protected]> * Add playback position to scrobble NowPlaying (#4089) * test(playtracker): cover playback position * address review comment Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> * fix merge Signed-off-by: Deluan <[email protected]> * refactor: remove unnecessary check for empty slice in Map function Signed-off-by: Deluan <[email protected]> * fix: update reflex.conf to include .wasm file extension Signed-off-by: Deluan <[email protected]> * fix(scanner): normalize attribute strings and add edge case tests for PID calculation Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458 Signed-off-by: Deluan <[email protected]> * test(ui): fix warnings (#4187) * fix(ui): address test warnings * ignore lint error in test Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> * refactor(server): optimize top songs lookup (#4189) * optimize top songs lookup * Optimize title matching queries * refactor: simplify top songs matching * improve error handling and logging in track loading functions Signed-off-by: Deluan <[email protected]> * test: add cases for fallback to title matching and combined MBID/title matching Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> * fix(ui): playlist details overflow in spotify-based themes (#4184) * test: ensure playlist details width * fix(test): simplify expectation for minWidth in NDPlaylistDetails Signed-off-by: Deluan <[email protected]> * fix(test): test all themes Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> * chore(deps): update TagLib to version 2.1 (#4185) * chore: update cross-taglib * fix(taglib): add logging for TagLib version Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> * test: verify agents fallback (#4191) * build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version. This is to reduce the image size, as we don't really need the latest. Signed-off-by: Deluan <[email protected]> * fix tests Signed-off-by: Deluan <[email protected]> * feat(runtime): implement pooled WASM runtime and module for better instance management Signed-off-by: Deluan <[email protected]> * fix(discord-plugin): adjust timer delay calculation for track completion Signed-off-by: Deluan <[email protected]> * resolve PR comments Signed-off-by: Deluan <[email protected]> * feat(plugins): implement cache cleanup by size functionality Signed-off-by: Deluan <[email protected]> * fix(manager): return error from getCompilationCache and handle it in ScanPlugins Signed-off-by: Deluan <[email protected]> * fix possible rce condition Signed-off-by: Deluan <[email protected]> * feat(docs): update README to include Cache and Artwork services Signed-off-by: Deluan <[email protected]> * feat(manager): add permissions support for host services in custom runtime - WIP Signed-off-by: Deluan <[email protected]> * feat(manifest): add permissions field to plugin manifests - WIP Signed-off-by: Deluan <[email protected]> * test(permissions): implement permission validation and testing for plugins - WIP Signed-off-by: Deluan <[email protected]> * feat(plugins): add unauthorized_plugin to test permission enforcement - WIP Signed-off-by: Deluan <[email protected]> * feat(docs): add Plugin Permission System section to README - WIP Signed-off-by: Deluan <[email protected]> * feat(manifest): add detailed reasons for permissions in plugin manifests - WIP Signed-off-by: Deluan <[email protected]> * feat(permissions): implement granular HTTP permissions for plugins - WIP Signed-off-by: Deluan <[email protected]> * feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP Signed-off-by: Deluan <[email protected]> * refactor Signed-off-by: Deluan <[email protected]> * refactor: unexport all plugins package private symbols Signed-off-by: Deluan <[email protected]> * update docs Signed-off-by: Deluan <[email protected]> * refactor: rename plugin_lifecycle_manager Signed-off-by: Deluan <[email protected]> * docs: add discord-rich-presence plugin example to README Signed-off-by: Deluan <[email protected]> * feat: add support for PATCH, HEAD, and OPTIONS HTTP methods Signed-off-by: Deluan <[email protected]> * feat: use folder names as unique identifiers for plugins Signed-off-by: Deluan <[email protected]> * fix: read config just once, to avoid data race in tests Signed-off-by: Deluan <[email protected]> * refactor: rename pluginName to pluginID for consistency across services Signed-off-by: Deluan <[email protected]> * fix: use symlink name instead of folder name for plugin registration Signed-off-by: Deluan <[email protected]> * feat: update plugin output format to include ID and enhance README with symlink usage Signed-off-by: Deluan <[email protected]> * refactor: implement shared plugin discovery function to streamline plugin scanning and error handling Signed-off-by: Deluan <[email protected]> * feat: show plugin permissions in `plugin info` Signed-off-by: Deluan <[email protected]> * feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP Signed-off-by: Deluan <[email protected]> * feat: implement typed permissions for plugins to enhance permission handling Signed-off-by: Deluan <[email protected]> * feat: refactor plugin permissions to use typed schema and improve validation - WIP Signed-off-by: Deluan <[email protected]> * feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP Signed-off-by: Deluan <[email protected]> * feat: remove unused JSON schema validation for plugin manifests Signed-off-by: Deluan <[email protected]> * feat: remove unused fields from PluginPackage struct in package.go Signed-off-by: Deluan <[email protected]> * feat: update file permissions in tests and remove unused permission parsing function Signed-off-by: Deluan <[email protected]> * feat: refactor test plugin creation to use typed permissions and remove legacy helper Signed-off-by: Deluan <[email protected]> * feat: add website field to plugin manifests and update test cases Signed-off-by: Deluan <[email protected]> * refactor: permission schema to use basePermission structure for consistency Signed-off-by: Deluan <[email protected]> * feat: enhance host service management by adding permission checks for each service Signed-off-by: Deluan <[email protected]> * refactor: reorganize code files Signed-off-by: Deluan <[email protected]> * refactor: simplify custom runtime creation by removing compilation cache parameter Signed-off-by: Deluan <[email protected]> * doc: add WebSocketService and update ConfigService for plugin-specific configuration Signed-off-by: Deluan <[email protected]> * feat: implement WASM loading optimization to enhance plugin instance creation speed Signed-off-by: Deluan <[email protected]> * refactor: rename custom runtime functions and update related tests for clarity Signed-off-by: Deluan <[email protected]> * refactor: enhance plugin structure with compilation handling and error reporting Signed-off-by: Deluan <[email protected]> * refactor: improve logging and context tracing in runtime and wasm base plugin Signed-off-by: Deluan <[email protected]> * refactor: enhance runtime management with scoped runtime and caching improvements Signed-off-by: Deluan <[email protected]> * refactor: implement EnsureCompiled method for improved plugin compilation handling Signed-off-by: Deluan <[email protected]> * refactor: implement cached module management with TTL for improved performance Signed-off-by: Deluan <[email protected]> * refactor: replace map with sync.Map Signed-off-by: Deluan <[email protected]> * refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness Signed-off-by: Deluan <[email protected]> * refactor: enhance image processing with fallback mechanism for improved error handling Signed-off-by: Deluan <[email protected]> * docs: review test plugins readme Signed-off-by: Deluan <[email protected]> * feat: set default timeout for HTTP client to 10 seconds Signed-off-by: Deluan <[email protected]> * feat: enhance wasm instance pool with concurrency limits and timeout settings Signed-off-by: Deluan <[email protected]> * feat(discordrp): implement caching for processed image URLs with configurable TTL Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
||
|
|
c795bcfcf7 |
feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709)
* fix(server): more race conditions when updating artist/album from external sources Signed-off-by: Deluan <[email protected]> * feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394 Signed-off-by: Deluan <[email protected]> * fix(ui): null Signed-off-by: Deluan <[email protected]> * fix(scanner): pass configfile option to child process Signed-off-by: Deluan <[email protected]> * fix(scanner): resume interrupted fullScans Signed-off-by: Deluan <[email protected]> * fix(scanner): remove old scanner code Signed-off-by: Deluan <[email protected]> * fix(scanner): rename old metadata package Signed-off-by: Deluan <[email protected]> * fix(scanner): move old metadata package Signed-off-by: Deluan <[email protected]> * fix: tests Signed-off-by: Deluan <[email protected]> * chore(deps): update Go to 1.23.4 Signed-off-by: Deluan <[email protected]> * fix: logs Signed-off-by: Deluan <[email protected]> * fix(test): Signed-off-by: Deluan <[email protected]> * fix: log level Signed-off-by: Deluan <[email protected]> * fix: remove log message Signed-off-by: Deluan <[email protected]> * feat: add config for scanner watcher Signed-off-by: Deluan <[email protected]> * refactor: children playlists Signed-off-by: Deluan <[email protected]> * refactor: replace `interface{}` with `any` Signed-off-by: Deluan <[email protected]> * fix: smart playlists with genres Signed-off-by: Deluan <[email protected]> * fix: allow any tags in smart playlists Signed-off-by: Deluan <[email protected]> * fix: artist names in playlists Signed-off-by: Deluan <[email protected]> * fix: smart playlist's sort by tags Signed-off-by: Deluan <[email protected]> * feat(subsonic): add moods to child Signed-off-by: Deluan <[email protected]> * feat(subsonic): add moods to AlbumID3 Signed-off-by: Deluan <[email protected]> * refactor(subsonic): use generic JSONArray for OS arrays Signed-off-by: Deluan <[email protected]> * refactor(subsonic): use https in test Signed-off-by: Deluan <[email protected]> * feat(subsonic): add releaseTypes to AlbumID3 Signed-off-by: Deluan <[email protected]> * feat(subsonic): add recordLabels to AlbumID3 Signed-off-by: Deluan <[email protected]> * refactor(subsonic): rename JSONArray to Array Signed-off-by: Deluan <[email protected]> * feat(subsonic): add artists to AlbumID3 Signed-off-by: Deluan <[email protected]> * feat(subsonic): add artists to Child Signed-off-by: Deluan <[email protected]> * fix(scanner): do not pre-populate smart playlists Signed-off-by: Deluan <[email protected]> * feat(subsonic): implement a simplified version of ArtistID3. See https://github.com/opensubsonic/open-subsonic-api/discussions/120 Signed-off-by: Deluan <[email protected]> * feat(subsonic): add artists to album child Signed-off-by: Deluan <[email protected]> * feat(subsonic): add contributors to mediafile Child Signed-off-by: Deluan <[email protected]> * feat(subsonic): add albumArtists to mediafile Child Signed-off-by: Deluan <[email protected]> * feat(subsonic): add displayArtist and displayAlbumArtist Signed-off-by: Deluan <[email protected]> * feat(subsonic): add displayComposer to Child Signed-off-by: Deluan <[email protected]> * feat(subsonic): add roles to ArtistID3 Signed-off-by: Deluan <[email protected]> * fix(subsonic): use " • " separator for displayComposer Signed-off-by: Deluan <[email protected]> * refactor: Signed-off-by: Deluan <[email protected]> * fix(subsonic): Signed-off-by: Deluan <[email protected]> * fix(subsonic): respect `PreferSortTags` config option Signed-off-by: Deluan <[email protected]> * refactor(subsonic): Signed-off-by: Deluan <[email protected]> * refactor: optimize purging non-unused tags Signed-off-by: Deluan <[email protected]> * refactor: don't run 'refresh artist stats' concurrently with other transactions Signed-off-by: Deluan <[email protected]> * refactor: Signed-off-by: Deluan <[email protected]> * fix: log message Signed-off-by: Deluan <[email protected]> * feat: add Scanner.ScanOnStartup config option, default true Signed-off-by: Deluan <[email protected]> * feat: better json parsing error msg when importing NSPs Signed-off-by: Deluan <[email protected]> * fix: don't update album's imported_time when updating external_metadata Signed-off-by: Deluan <[email protected]> * fix: handle interrupted scans and full scans after migrations Signed-off-by: Deluan <[email protected]> * feat: run `analyze` when migration requires a full rescan Signed-off-by: Deluan <[email protected]> * feat: run `PRAGMA optimize` at the end of the scan Signed-off-by: Deluan <[email protected]> * fix: don't update artist's updated_at when updating external_metadata Signed-off-by: Deluan <[email protected]> * feat: handle multiple artists and roles in smart playlists Signed-off-by: Deluan <[email protected]> * feat(ui): dim missing tracks Signed-off-by: Deluan <[email protected]> * fix: album missing logic Signed-off-by: Deluan <[email protected]> * fix: error encoding in gob Signed-off-by: Deluan <[email protected]> * feat: separate warnings from errors Signed-off-by: Deluan <[email protected]> * fix: mark albums as missing if they were contained in a deleted folder Signed-off-by: Deluan <[email protected]> * refactor: add participant names to media_file and album tables Signed-off-by: Deluan <[email protected]> * refactor: use participations in criteria, instead of m2m relationship Signed-off-by: Deluan <[email protected]> * refactor: rename participations to participants Signed-off-by: Deluan <[email protected]> * feat(subsonic): add moods to album child Signed-off-by: Deluan <[email protected]> * fix: albumartist role case Signed-off-by: Deluan <[email protected]> * feat(scanner): run scanner as an external process by default Signed-off-by: Deluan <[email protected]> * fix(ui): show albumArtist names Signed-off-by: Deluan <[email protected]> * fix(ui): dim out missing albums Signed-off-by: Deluan <[email protected]> * fix: flaky test Signed-off-by: Deluan <[email protected]> * fix(server): scrobble buffer mapping. fix #3583 Signed-off-by: Deluan <[email protected]> * refactor: more participations renaming Signed-off-by: Deluan <[email protected]> * fix: listenbrainz scrobbling Signed-off-by: Deluan <[email protected]> * feat: send release_group_mbid to listenbrainz Signed-off-by: Deluan <[email protected]> * feat(subsonic): implement OpenSubsonic explicitStatus field (#3597) * feat: implement OpenSubsonic explicitStatus field * fix(subsonic): fix failing snapshot tests * refactor: create helper for setting explicitStatus * fix: store smaller values for explicit-status on database * test: ToAlbum explicitStatus * refactor: rename explicitStatus helper function --------- Co-authored-by: Deluan Quintão <[email protected]> * fix: handle album and track tags in the DB based on the mappings.yaml file Signed-off-by: Deluan <[email protected]> * save similar artists as JSONB Signed-off-by: Deluan <[email protected]> * fix: getAlbumList byGenre Signed-off-by: Deluan <[email protected]> * detect changes in PID configuration Signed-off-by: Deluan <[email protected]> * set default album PID to legacy_pid Signed-off-by: Deluan <[email protected]> * fix tests Signed-off-by: Deluan <[email protected]> * fix SIGSEGV Signed-off-by: Deluan <[email protected]> * fix: don't lose album stars/ratings when migrating Signed-off-by: Deluan <[email protected]> * store full PID conf in properties Signed-off-by: Deluan <[email protected]> * fix: keep album annotations when changing PID.Album config Signed-off-by: Deluan <[email protected]> * fix: reassign album annotations Signed-off-by: Deluan <[email protected]> * feat: use (display) albumArtist and add links to each artist Signed-off-by: Deluan <[email protected]> * fix: not showing albums by albumartist Signed-off-by: Deluan <[email protected]> * fix: error msgs Signed-off-by: Deluan <[email protected]> * fix: hide PID from Native API Signed-off-by: Deluan <[email protected]> * fix: album cover art resolution Signed-off-by: Deluan <[email protected]> * fix: trim participant names Signed-off-by: Deluan <[email protected]> * fix: reduce watcher log spam Signed-off-by: Deluan <[email protected]> * fix: panic when initializing the watcher Signed-off-by: Deluan <[email protected]> * fix: various artists Signed-off-by: Deluan <[email protected]> * fix: don't store empty lyrics in the DB Signed-off-by: Deluan <[email protected]> * remove unused methods Signed-off-by: Deluan <[email protected]> * drop full_text indexes, as they are not being used by SQLite Signed-off-by: Deluan <[email protected]> * keep album created_at when upgrading Signed-off-by: Deluan <[email protected]> * fix(ui): null pointer Signed-off-by: Deluan <[email protected]> * fix: album artwork cache Signed-off-by: Deluan <[email protected]> * fix: don't expose missing files in Subsonic API Signed-off-by: Deluan <[email protected]> * refactor: searchable interface Signed-off-by: Deluan <[email protected]> * fix: filter out missing items from subsonic search * fix: filter out missing items from playlists * fix: filter out missing items from shares Signed-off-by: Deluan <[email protected]> * feat(ui): add filter by artist role Signed-off-by: Deluan <[email protected]> * feat(subsonic): only return albumartists in getIndexes and getArtists endpoints Signed-off-by: Deluan <[email protected]> * sort roles alphabetically Signed-off-by: Deluan <[email protected]> * fix: artist playcounts Signed-off-by: Deluan <[email protected]> * change default Album PID conf Signed-off-by: Deluan <[email protected]> * fix albumartist link when it does not match any albumartists values Signed-off-by: Deluan <[email protected]> * fix `Ignoring filter not whitelisted` (role) message Signed-off-by: Deluan <[email protected]> * fix: trim any names/titles being imported Signed-off-by: Deluan <[email protected]> * remove unused genre code Signed-off-by: Deluan <[email protected]> * serialize calls to Last.fm's getArtist Signed-off-by: Deluan <[email protected]> xxx Signed-off-by: Deluan <[email protected]> * add counters to genres Signed-off-by: Deluan <[email protected]> * nit: fix migration `notice` message Signed-off-by: Deluan <[email protected]> * optimize similar artists query Signed-off-by: Deluan <[email protected]> * fix: last.fm.getInfo when mbid does not exist Signed-off-by: Deluan <[email protected]> * ui only show missing items for admins Signed-off-by: Deluan <[email protected]> * don't allow interaction with missing items Signed-off-by: Deluan <[email protected]> * Add Missing Files view (WIP) Signed-off-by: Deluan <[email protected]> * refactor: merged tag_counts into tag table Signed-off-by: Deluan <[email protected]> * add option to completely disable automatic scanner Signed-off-by: Deluan <[email protected]> * add delete missing files functionality Signed-off-by: Deluan <[email protected]> * fix: playlists not showing for regular users Signed-off-by: Deluan <[email protected]> * reduce updateLastAccess frequency to once every minute Signed-off-by: Deluan <[email protected]> * reduce update player frequency to once every minute Signed-off-by: Deluan <[email protected]> * add timeout when updating player Signed-off-by: Deluan <[email protected]> * remove dead code Signed-off-by: Deluan <[email protected]> * fix duplicated roles in stats Signed-off-by: Deluan <[email protected]> * add `; ` to artist splitters Signed-off-by: Deluan <[email protected]> * fix stats query Signed-off-by: Deluan <[email protected]> * more logs Signed-off-by: Deluan <[email protected]> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <[email protected]> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <[email protected]> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <[email protected]> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <[email protected]> * add record label filter Signed-off-by: Deluan <[email protected]> * add release type filter Signed-off-by: Deluan <[email protected]> * fix purgeUnused tags Signed-off-by: Deluan <[email protected]> * add grouping filter to albums Signed-off-by: Deluan <[email protected]> * allow any album tags to be used in as filters in the API Signed-off-by: Deluan <[email protected]> * remove empty tags from album info Signed-off-by: Deluan <[email protected]> * comments in the migration Signed-off-by: Deluan <[email protected]> * fix: Cannot read properties of undefined Signed-off-by: Deluan <[email protected]> * fix: listenbrainz scrobbling (#3640) Signed-off-by: Deluan <[email protected]> * fix: remove duplicated tag values Signed-off-by: Deluan <[email protected]> * fix: don't ignore the taglib folder! Signed-off-by: Deluan <[email protected]> * feat: show track subtitle tag Signed-off-by: Deluan <[email protected]> * fix: show artists stats based on selected role Signed-off-by: Deluan <[email protected]> * fix: inspect Signed-off-by: Deluan <[email protected]> * add media type to album info/filters Signed-off-by: Deluan <[email protected]> * fix: change format of subtitle in the UI Signed-off-by: Deluan <[email protected]> * fix: subtitle in Subsonic API and search Signed-off-by: Deluan <[email protected]> * fix: subtitle in UI's player Signed-off-by: Deluan <[email protected]> * fix: split strings should be case-insensitive Signed-off-by: Deluan <[email protected]> * disable ScanSchedule Signed-off-by: Deluan <[email protected]> * increase default sessiontimeout Signed-off-by: Deluan <[email protected]> * add sqlite command line tool to docker image Signed-off-by: Deluan <[email protected]> * fix: resources override Signed-off-by: Deluan <[email protected]> * fix: album PID conf Signed-off-by: Deluan <[email protected]> * change migration to mark current artists as albumArtists Signed-off-by: Deluan <[email protected]> * feat(ui): Allow filtering on multiple genres (#3679) * feat(ui): Allow filtering on multiple genres Signed-off-by: Henrik Nordvik <[email protected]> Signed-off-by: Deluan <[email protected]> * add multi-genre filter in Album list Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Henrik Nordvik <[email protected]> Signed-off-by: Deluan <[email protected]> Co-authored-by: Henrik Nordvik <[email protected]> * add more multi-valued tag filters to Album and Song views Signed-off-by: Deluan <[email protected]> * fix(ui): unselect missing files after removing Signed-off-by: Deluan <[email protected]> * fix(ui): song filter Signed-off-by: Deluan <[email protected]> * fix sharing tracks. fix #3687 Signed-off-by: Deluan <[email protected]> * use rowids when using search for sync (ex: Symfonium) Signed-off-by: Deluan <[email protected]> * fix "Report Real Paths" option for subsonic clients Signed-off-by: Deluan <[email protected]> * fix "Report Real Paths" option for subsonic clients for search Signed-off-by: Deluan <[email protected]> * add libraryPath to Native API /songs endpoint Signed-off-by: Deluan <[email protected]> * feat(subsonic): add album version Signed-off-by: Deluan <[email protected]> * made all tags lowercase as they are case-insensitive anyways. Signed-off-by: Deluan <[email protected]> * feat(ui): Show full paths, extended properties for album/song (#3691) * feat(ui): Show full paths, extended properties for album/song - uses library path + os separator + path - show participants (album/song) and tags (song) - make album/participant clickable in show info * add source to path * fix pathSeparator in UI Signed-off-by: Deluan <[email protected]> * fix local artist artwork (#3695) Signed-off-by: Deluan <[email protected]> * fix: parse vorbis performers Signed-off-by: Deluan <[email protected]> * refactor: clean function into smaller functions Signed-off-by: Deluan <[email protected]> * fix translations for en and pt Signed-off-by: Deluan <[email protected]> * add trace log to show annotations reassignment Signed-off-by: Deluan <[email protected]> * add trace log to show annotations reassignment Signed-off-by: Deluan <[email protected]> * fix: allow performers without instrument/subrole Signed-off-by: Deluan <[email protected]> * refactor: metadata clean function again Signed-off-by: Deluan <[email protected]> * refactor: optimize split function Signed-off-by: Deluan <[email protected]> * refactor: split function is now a method of TagConf Signed-off-by: Deluan <[email protected]> * fix: humanize Artist total size Signed-off-by: Deluan <[email protected]> * add album version to album details Signed-off-by: Deluan <[email protected]> * don't display album-level tags in SongInfo Signed-off-by: Deluan <[email protected]> * fix genre clicking in Album Page Signed-off-by: Deluan <[email protected]> * don't use mbids in Last.fm api calls. From https://discord.com/channels/671335427726114836/704303730660737113/1337574018143879248: With MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo { artist: { name: "Bee Gees", mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810", url: "https://www.last.fm/music/Bee+Gees", } ``` Without MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo { artist: { name: "Van Morrison", mbid: "a41ac10f-0a56-4672-9161-b83f9b223559", url: "https://www.last.fm/music/Van+Morrison", } ``` Signed-off-by: Deluan <[email protected]> * better logging for when the artist folder is not found Signed-off-by: Deluan <[email protected]> * fix various issues with artist image resolution Signed-off-by: Deluan <[email protected]> * hide "Additional Tags" header if there are none. Signed-off-by: Deluan <[email protected]> * simplify tag rendering Signed-off-by: Deluan <[email protected]> * enhance logging for artist folder detection Signed-off-by: Deluan <[email protected]> * make folderID consistent for relative and absolute folderPaths Signed-off-by: Deluan <[email protected]> * handle more folder paths scenarios Signed-off-by: Deluan <[email protected]> * filter out other roles when SubsonicArtistParticipations = true Signed-off-by: Deluan <[email protected]> * fix "Cannot read properties of undefined" Signed-off-by: Deluan <[email protected]> * fix lyrics and comments being truncated (#3701) * fix lyrics and comments being truncated * specifically test for lyrics and comment length * reorder assertions Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> Co-authored-by: Deluan <[email protected]> * fix(server): Expose library_path for playlist (#3705) Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic) * fix BFR on Windows (#3704) * fix potential reflected cross-site scripting vulnerability Signed-off-by: Deluan <[email protected]> * hack to make it work on Windows * ignore windows executables * try fixing the pipeline Signed-off-by: Deluan <[email protected]> * allow MusicFolder in other drives * move windows local drive logic to local storage implementation --------- Signed-off-by: Deluan <[email protected]> * increase pagination sizes for missing files Signed-off-by: Deluan <[email protected]> * reduce level of "already scanning" watcher log message Signed-off-by: Deluan <[email protected]> * only count folders with audio files in it See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930 Signed-off-by: Deluan <[email protected]> * add album version and catalog number to search Signed-off-by: Deluan <[email protected]> * add `organization` alias for `recordlabel` Signed-off-by: Deluan <[email protected]> * remove mbid from Last.fm agent Signed-off-by: Deluan <[email protected]> * feat: support inspect in ui (#3726) * inspect in ui * address round 1 * add catalogNum to AlbumInfo Signed-off-by: Deluan <[email protected]> * remove dependency on metadata_old (deprecated) package Signed-off-by: Deluan <[email protected]> * add `RawTags` to model Signed-off-by: Deluan <[email protected]> * support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698) * parse standard roles, vorbis/m4a work for now * fix djmixer * working roles, use DJ-mix * add performers to file * map mbids * add a few more tests * add test Signed-off-by: Deluan <[email protected]> * try to simplify the performers logic Signed-off-by: Deluan <[email protected]> * stylistic changes --------- Signed-off-by: Deluan <[email protected]> Co-authored-by: Deluan <[email protected]> * remove param mutation Signed-off-by: Deluan <[email protected]> * run automated SQLite optimizations Signed-off-by: Deluan <[email protected]> * fix playlists import/export on Windows * fix import playlists * fix export playlists * better handling of Windows volumes Signed-off-by: Deluan <[email protected]> * handle more album ID reassignments Signed-off-by: Deluan <[email protected]> * allow adding/overriding tags in the config file Signed-off-by: Deluan <[email protected]> * fix(ui): Fix playlist track id, handle missing tracks better (#3734) - Use `mediaFileId` instead of `id` for playlist tracks - Only fetch if the file is not missing - If extractor fails to get the file, also error (rather than panic) * optimize DB after each scan. Signed-off-by: Deluan <[email protected]> * remove sortable from AlbumSongs columns Signed-off-by: Deluan <[email protected]> * simplify query to get missing tracks Signed-off-by: Deluan <[email protected]> * mark Scanner.Extractor as deprecated Signed-off-by: Deluan <[email protected]> --------- Signed-off-by: Deluan <[email protected]> Signed-off-by: Henrik Nordvik <[email protected]> Co-authored-by: Caio Cotts <[email protected]> Co-authored-by: Henrik Nordvik <[email protected]> Co-authored-by: Kendall Garner <[email protected]> |
||
|
Deluan
|
6f7b48202e | Make the GetInstance concurrent test more readable | ||
|
Deluan
|
0d8f8e3afd | Optimize Singleton (sometimes a simple lock is a better solution) | ||
|
Deluan
|
7bafbce816 | Reduce number of goroutines in test, to avoid hitting the hard limit of 8128 | ||
|
Deluan
|
d613b19306 | Simplify Singleton usage by leveraging Go 1.18's generics | ||
|
Deluan
|
31882abf6f | Upgrade Ginkgo to V2 | ||
|
Deluan
|
25db2cb075 | Add concurrency test for singleton | ||
|
Deluan
|
f8ee6db72a | New implementation of NowPlaying |