diff --git a/.gitea/workflows/song-server.yml b/.gitea/workflows/song-server.yml new file mode 100644 index 00000000..421bf98d --- /dev/null +++ b/.gitea/workflows/song-server.yml @@ -0,0 +1,47 @@ +# Builds the song server image and pushes it to the Gitea registry: +# git.disi.dev/projects/vocaluxe-song-server: and :latest +# Release: set VocaluxeSongServer/VERSION, then push the tag songserver-v (e.g. songserver-v0.1.0). +# Manual runs (workflow_dispatch) push :dev only. +name: Song server image + +env: + DOCKER_API_VERSION: '1.43' + IMAGE: git.disi.dev/projects/vocaluxe-song-server + +on: + push: + tags: + - 'songserver-v*' + workflow_dispatch: + +jobs: + image: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v5 + - name: Determine tags + id: tags + run: | + VERSION=$(tr -d '[:space:]' < VocaluxeSongServer/VERSION) + if [ "${GITHUB_REF_TYPE}" = "tag" ]; then + if [ "${GITHUB_REF_NAME}" != "songserver-v${VERSION}" ]; then + echo "Tag ${GITHUB_REF_NAME} does not match VocaluxeSongServer/VERSION (${VERSION})" + exit 1 + fi + echo "tags=${VERSION} latest" >> "$GITHUB_OUTPUT" + else + echo "tags=dev" >> "$GITHUB_OUTPUT" + fi + - name: Login to Gitea container registry + uses: docker/login-action@v3 + with: + username: gitearobot + password: ${{ secrets.PACKAGE_GITEA_PAT }} + registry: git.disi.dev + - name: Build and publish + run: | + docker build -t "${IMAGE}:build" -f VocaluxeSongServer/Dockerfile . + for TAG in ${{ steps.tags.outputs.tags }}; do + docker tag "${IMAGE}:build" "${IMAGE}:${TAG}" + docker push "${IMAGE}:${TAG}" + done diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..74932df8 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,48 @@ +# Vocaluxe (fork at git.disi.dev/Projects/Vocaluxe) + +Fork of [Vocaluxe/Vocaluxe](https://github.com/Vocaluxe/Vocaluxe), a karaoke game (C#, .NET 10). Default branch +`develop` tracks upstream; the fork's own work happens on feature branches. Keep changes to upstream code small and +hook-based, so upstream can still be merged. + +## Goal of the fork +1. **Song server** (`VocaluxeSongServer/`, in progress): a central song library on the homelab. The game becomes the + frontend and downloads song files on demand. See `VocaluxeSongServer/README.md`. +2. Later: serve the guest WebApp ("browse by phone", `WebApp/` + `VocaluxeWeb/`) directly from the server. +3. Later: search and download songs that aren't in the library, from common internet sources (possibly UltraSinger + to create songs from audio). + +## Layout +| Folder | What | +|---|---| +| `Vocaluxe/` | The game. Song server client: `Base/SongServer/` (`CSongServer` sync, `CRemoteSongFileProvider` downloads) | +| `VocaluxeLib/` | Shared library. `Songs/CSongFiles.cs` is the hook every song file access goes through | +| `VocaluxeSongServer/` | ASP.NET Core server, Docker image `git.disi.dev/projects/vocaluxe-song-server` | +| `VocaluxeWeb/`, `WebApp/` | Built-in webserver of the game and the guest WebApp (Vue), upstream | +| `Tests/` | NUnit; song server tests in `Tests/SongServer` | + +## How the song server integration works +- `CSongs.LoadSongs` calls `CSongServer.Sync()`: fetches `/api/v1/library` (ETag), mirrors the song `.txt` files into + `/SongServer/_` (zip bundles), deletes files no longer on the server, and installs + `CRemoteSongFileProvider` as `CSongFiles.Provider`. The cache folder is then loaded like a normal song folder. +- `CSongFiles.Exists` (loader header checks) is true for files that only exist on the server. `IsLocal` means the file + is downloaded and current (size and mtime match the server). +- Downloads on demand: covers (`CSong.LoadSmallCover`, background threads, blocking), preview audio + (`CBackgroundMusic.LoadPreview` + `Update`), everything before singing (`CScreenSing._LoadAndStartCurrentSong`, + progress via `TR_SCREENSING_DOWNLOADING`). Downloads resume from `.part` files with Range requests. +- Config: `config.xml` section `SongServer` (`Url`, `ApiKey`). The contracts (`VocaluxeSongServer/Contracts`) are + linked into `Vocaluxe.csproj`; the test project references the server with `extern alias songserver`. + +## Build and test (Windows) +- `dotnet build Vocaluxe/Vocaluxe.csproj -c DebugWin -p:Platform=x64` (the whole solution also builds the native + `PitchTracker`, which needs Visual Studio's C++ tools). +- `dotnet test Tests/Tests.csproj -c DebugWin -p:Platform=x64` +- Building rewrites the copyright year in `Vocaluxe/Properties/AssemblyInfo.cs`; don't commit that. +- Don't use `dotnet sln add`: it rewrites the configuration table of `Vocaluxe.sln`. Copy the lines of `VocaluxeWeb`. +- New `config.xml` fields must also go into `Tests/VocaluxeLib/XML/TestFiles/SConfig.xml` (round-trip test; CRLF). + +## Known gaps (song server) +- Cover/background auto-detection (`[CO]`/`[BG]` files without a tag, `CSong._CheckFiles`) only sees local files. +- Media in subfolders of a song folder are not served (only the song folder itself). +- On Linux clients, media whose name differs in case from the `.txt` reference won't be found after download. +- No cache size limit yet. The WebApp's cover images for server songs depend on the cover loader having fetched them. +- No in-game UI for the server address yet (only `config.xml`). diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..43c994c2 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md diff --git a/Output/Languages/en.xml b/Output/Languages/en.xml index 7c0c3c6b..82aa9647 100644 --- a/Output/Languages/en.xml +++ b/Output/Languages/en.xml @@ -267,6 +267,7 @@ Restart game Vocals track volume No more songs in playlist! + Downloading %s ... %p% No webcam available! Aspect Ratio: %a perfect! diff --git a/Tests/SongServer/CSongServerTest.cs b/Tests/SongServer/CSongServerTest.cs new file mode 100644 index 00000000..3f6e1c70 --- /dev/null +++ b/Tests/SongServer/CSongServerTest.cs @@ -0,0 +1,214 @@ +#region license +// This file is part of Vocaluxe. +// +// Vocaluxe is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Vocaluxe is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with Vocaluxe. If not, see . +#endregion + +extern alias songserver; + +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Net; +using System.Net.Http; +using System.Text; +using System.Text.Json; +using System.Threading.Tasks; +using Microsoft.AspNetCore.Hosting; +using Microsoft.AspNetCore.Mvc.Testing; +using Microsoft.Extensions.Logging.Abstractions; +using NUnit.Framework; +using Vocaluxe.Base.SongServer; +using VocaluxeSongServer.Contracts; +using CLibraryScanner = songserver::VocaluxeSongServer.Library.CLibraryScanner; +using ServerProgram = songserver::Program; + +namespace Tests.SongServer +{ + [TestFixture] + public class CSongServerTest + { + private string _Root; + private string _Library; + private string _Cache; + + [SetUp] + public void SetUp() + { + _Root = Path.Combine(Path.GetTempPath(), "vlx-songserver-test-" + Guid.NewGuid().ToString("N")); + _Library = Path.Combine(_Root, "library"); + _Cache = Path.Combine(_Root, "cache"); + Directory.CreateDirectory(Path.Combine(_Library, "A - One")); + Directory.CreateDirectory(Path.Combine(_Library, "Sub", "Künstler - Lied")); + Directory.CreateDirectory(Path.Combine(_Library, "NoSong")); + File.WriteAllText(Path.Combine(_Library, "A - One", "A - One.txt"), "#TITLE:One\n#ARTIST:A\n#MP3:One.mp3\n#COVER:cover.jpg\n#BPM:300\n: 0 4 5 La\nE\n"); + File.WriteAllBytes(Path.Combine(_Library, "A - One", "One.mp3"), _RandomBytes(300000, 1)); + File.WriteAllBytes(Path.Combine(_Library, "A - One", "cover.jpg"), _RandomBytes(2000, 2)); + // Latin-1 encoded, like many old song files + File.WriteAllBytes(Path.Combine(_Library, "Sub", "Künstler - Lied", "lied.txt"), + Encoding.Latin1.GetBytes("#TITLE:Lied über\n#ARTIST:Künstler\n#AUDIO:a.ogg\n#BPM:200\n: 0 4 5 La\nE\n")); + File.WriteAllBytes(Path.Combine(_Library, "Sub", "Künstler - Lied", "a.ogg"), _RandomBytes(1000, 3)); + File.WriteAllText(Path.Combine(_Library, "NoSong", "readme.txt"), "not a song"); + } + + [TearDown] + public void TearDown() + { + try + { + Directory.Delete(_Root, true); + } + catch (IOException) { } + } + + private static byte[] _RandomBytes(int count, int seed) + { + var bytes = new byte[count]; + new Random(seed).NextBytes(bytes); + return bytes; + } + + [Test] + public void TestParseHeader() + { + var header = CLibraryScanner.ParseHeader("#TITLE:Song\r\n#ARTIST:Artist\r\n#MP3:x.mp3\r\n: 0 1 1 a\r\n#TITLE:ignored\n"); + Assert.That(header, Is.Not.Null); + Assert.That(header.Title, Is.EqualTo("Song")); + Assert.That(header.Artist, Is.EqualTo("Artist")); + Assert.That(CLibraryScanner.ParseHeader("#TITLE:Song\n#ARTIST:Artist\n"), Is.Null, "no audio"); + Assert.That(CLibraryScanner.ParseHeader("just text"), Is.Null); + } + + [Test] + public void TestDecodeText() + { + Assert.That(CLibraryScanner.DecodeText(Encoding.UTF8.GetPreamble().Concat(Encoding.UTF8.GetBytes("über")).ToArray()), Is.EqualTo("über")); + Assert.That(CLibraryScanner.DecodeText(Encoding.UTF8.GetBytes("über")), Is.EqualTo("über")); + Assert.That(CLibraryScanner.DecodeText(Encoding.Latin1.GetBytes("über")), Is.EqualTo("über")); + } + + [Test] + public void TestScan() + { + var scanner = new CLibraryScanner(NullLogger.Instance); + var snapshot = scanner.Scan(_Library); + Assert.That(snapshot.Dto.Songs.Select(s => s.Path), Is.EqualTo(new[] { "A - One/A - One.txt", "Sub/Künstler - Lied/lied.txt" })); + Assert.That(snapshot.Dto.Songs[1].Title, Is.EqualTo("Lied über")); + Assert.That(snapshot.Dto.Files.Select(f => f.Path), Does.Not.Contain("NoSong/readme.txt"), "only files in song folders are served"); + Assert.That(snapshot.Dto.Files.Count, Is.EqualTo(5)); + + var version = snapshot.Dto.Version; + Assert.That(scanner.Scan(_Library).Dto.Version, Is.EqualTo(version), "unchanged library, same version"); + File.WriteAllBytes(Path.Combine(_Library, "A - One", "One.mp3"), _RandomBytes(300001, 1)); + Assert.That(scanner.Scan(_Library).Dto.Version, Is.Not.EqualTo(version), "changed file, new version"); + } + + private WebApplicationFactory _CreateServer(string apiKey = "") + { + return new WebApplicationFactory().WithWebHostBuilder(b => + { + b.UseSetting("Library:Path", _Library); + b.UseSetting("Library:ApiKey", apiKey); + }); + } + + private static async Task _GetLibrary(HttpClient client) + { + for (var i = 0; i < 50; i++) + { + var response = await client.GetAsync("/api/v1/library"); + if (response.StatusCode != HttpStatusCode.ServiceUnavailable) + { + response.EnsureSuccessStatusCode(); + return JsonSerializer.Deserialize(await response.Content.ReadAsStringAsync(), new JsonSerializerOptions(JsonSerializerDefaults.Web)); + } + + await Task.Delay(100); // first scan still running + } + + throw new TimeoutException("library scan did not finish"); + } + + [Test] + public async Task TestApiKey() + { + using (var server = _CreateServer("secret")) + { + var client = server.CreateClient(); + Assert.That((await client.GetAsync("/api/v1/library")).StatusCode, Is.EqualTo(HttpStatusCode.Unauthorized)); + client.DefaultRequestHeaders.Add("X-Api-Key", "secret"); + Assert.That((await _GetLibrary(client)).Songs.Count, Is.EqualTo(2)); + } + } + + [Test] + public async Task TestDownloadOnDemand() + { + using (var server = _CreateServer()) + { + var client = server.CreateClient(); + var library = await _GetLibrary(client); + var provider = new CRemoteSongFileProvider(client, _Cache, library.Files); + + var mp3 = CRemoteSongFileProvider.GetLocalPath(_Cache, "A - One/One.mp3"); + // Case differs from the server file, as often in song files + var cover = CRemoteSongFileProvider.GetLocalPath(_Cache, "A - One/COVER.jpg"); + var ogg = CRemoteSongFileProvider.GetLocalPath(_Cache, "Sub/Künstler - Lied/a.ogg"); + + Assert.That(provider.Exists(mp3), Is.True); + Assert.That(provider.IsLocal(mp3), Is.False); + Assert.That(provider.Exists(Path.Combine(_Cache, "nothing.mp3")), Is.False); + Assert.That(provider.GetProgress(new[] { mp3 }), Is.EqualTo(0f)); + + // Interrupted earlier download: the rest is fetched with a Range request + Directory.CreateDirectory(Path.GetDirectoryName(mp3)); + File.WriteAllBytes(mp3 + ".part", _RandomBytes(300000, 1).Take(1000).ToArray()); + + await provider.RequestAsync(new[] { mp3, cover, ogg, mp3 }); + + Assert.That(provider.IsLocal(mp3) && provider.IsLocal(cover) && provider.IsLocal(ogg), Is.True); + Assert.That(File.ReadAllBytes(mp3), Is.EqualTo(File.ReadAllBytes(Path.Combine(_Library, "A - One", "One.mp3")))); + Assert.That(File.ReadAllBytes(ogg), Is.EqualTo(File.ReadAllBytes(Path.Combine(_Library, "Sub", "Künstler - Lied", "a.ogg")))); + Assert.That(File.Exists(mp3 + ".part"), Is.False); + Assert.That(provider.GetProgress(new[] { mp3, cover, ogg }), Is.EqualTo(1f)); + + // A changed file on the server makes the local copy outdated + var changed = library.Files.Select(f => f.Path == "A - One/One.mp3" ? new FileDto { Path = f.Path, Size = f.Size, Modified = f.Modified + 60000 } : f); + Assert.That(new CRemoteSongFileProvider(client, _Cache, changed).IsLocal(mp3), Is.False); + } + } + + [Test] + public async Task TestBundleAndFileSecurity() + { + using (var server = _CreateServer()) + { + var client = server.CreateClient(); + await _GetLibrary(client); + var body = new StringContent(JsonSerializer.Serialize(new BundleRequest { Paths = new List { "Sub/Künstler - Lied/lied.txt", "NoSong/readme.txt" } }, + new JsonSerializerOptions(JsonSerializerDefaults.Web)), Encoding.UTF8, "application/json"); + var response = await client.PostAsync("/api/v1/files/bundle", body); + response.EnsureSuccessStatusCode(); + using (var zip = new System.IO.Compression.ZipArchive(await response.Content.ReadAsStreamAsync())) + { + Assert.That(zip.Entries.Select(e => e.FullName), Is.EqualTo(new[] { "Sub/Künstler - Lied/lied.txt" })); + } + + Assert.That((await client.GetAsync("/api/v1/files/NoSong/readme.txt")).StatusCode, Is.EqualTo(HttpStatusCode.NotFound)); + Assert.That((await client.GetAsync("/api/v1/files/..%2F..%2Fsecret.txt")).StatusCode, Is.EqualTo(HttpStatusCode.NotFound)); + } + } + } +} diff --git a/Tests/Tests.csproj b/Tests/Tests.csproj index cb452ec9..ef9d274c 100644 --- a/Tests/Tests.csproj +++ b/Tests/Tests.csproj @@ -15,6 +15,7 @@ + @@ -32,5 +33,7 @@ + + diff --git a/Tests/VocaluxeLib/XML/TestFiles/SConfig.xml b/Tests/VocaluxeLib/XML/TestFiles/SConfig.xml index ef09a8b2..08ae13a0 100644 --- a/Tests/VocaluxeLib/XML/TestFiles/SConfig.xml +++ b/Tests/VocaluxeLib/XML/TestFiles/SConfig.xml @@ -137,4 +137,8 @@ 3000 70 + + + + diff --git a/Vocaluxe.sln b/Vocaluxe.sln index fde0473b..1d2acba6 100644 --- a/Vocaluxe.sln +++ b/Vocaluxe.sln @@ -26,6 +26,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "PartyModeTournament", "Part EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "VocaluxeWeb", "VocaluxeWeb\VocaluxeWeb.csproj", "{4E2C9A71-6B3D-4F85-9C1E-8A7D5B2F3C60}" EndProject +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "VocaluxeSongServer", "VocaluxeSongServer\VocaluxeSongServer.csproj", "{9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}" +EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "PartyModeRuleRoulette", "PartyModes\PartyModeRuleRoulette\PartyModeRuleRoulette.csproj", "{2E6A9B48-71C3-4D05-8F2B-C4A7D19E6E3F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "PitchTracker", "PitchTracker\PitchTracker.vcxproj", "{54A48AF7-515C-4EDD-B395-2BFFA096050A}" @@ -264,6 +266,22 @@ Global {4E2C9A71-6B3D-4F85-9C1E-8A7D5B2F3C60}.ReleaseWin|x64.Build.0 = ReleaseWin|Any CPU {4E2C9A71-6B3D-4F85-9C1E-8A7D5B2F3C60}.ReleaseWin|x86.ActiveCfg = ReleaseWin|Any CPU {4E2C9A71-6B3D-4F85-9C1E-8A7D5B2F3C60}.ReleaseWin|x86.Build.0 = ReleaseWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.DebugLinux|x64.ActiveCfg = DebugLinux|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.DebugLinux|x86.ActiveCfg = DebugLinux|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.DebugWin|x64.ActiveCfg = DebugWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.DebugWin|x64.Build.0 = DebugWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.DebugWin|x86.ActiveCfg = DebugWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.DebugWin|x86.Build.0 = DebugWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.InstallerWin|x64.ActiveCfg = InstallerWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.InstallerWin|x64.Build.0 = InstallerWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.InstallerWin|x86.ActiveCfg = InstallerWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.InstallerWin|x86.Build.0 = InstallerWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.ReleaseLinux|x64.ActiveCfg = ReleaseLinux|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.ReleaseLinux|x86.ActiveCfg = ReleaseLinux|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.ReleaseWin|x64.ActiveCfg = ReleaseWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.ReleaseWin|x64.Build.0 = ReleaseWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.ReleaseWin|x86.ActiveCfg = ReleaseWin|Any CPU + {9595FC0E-E9A4-4775-9E6B-A5004BD2DA34}.ReleaseWin|x86.Build.0 = ReleaseWin|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE diff --git a/Vocaluxe/Base/CBackgroundMusic.cs b/Vocaluxe/Base/CBackgroundMusic.cs index 22574e3a..7f2e93ca 100644 --- a/Vocaluxe/Base/CBackgroundMusic.cs +++ b/Vocaluxe/Base/CBackgroundMusic.cs @@ -18,6 +18,7 @@ using System; using System.Collections.Generic; using System.Diagnostics; +using System.Linq; using System.Threading.Tasks; using VocaluxeLib; using VocaluxeLib.Draw; @@ -55,6 +56,11 @@ namespace Vocaluxe.Base private const int _PreviewStartWaitMaxTries = 10; private static readonly object _PreviewStartHelperTaskLock = new Object(); + // Song server: preview waiting for its audio file to be downloaded (main thread only) + private static CSong _PendingPreviewSong; + private static float _PendingPreviewStart; + private static Task _PendingPreviewTask; + public static bool VideoEnabled { get { return _CurPlayer.VideoEnabled; } @@ -309,6 +315,16 @@ namespace Vocaluxe.Base public static void Update() { + if (_PendingPreviewSong != null && _PendingPreviewTask.IsCompleted) + { + var song = _PendingPreviewSong; + _PendingPreviewSong = null; + if (CSongFiles.IsLocal(song.GetMP3())) + { + LoadPreview(song, _PendingPreviewStart); + } + } + if (_PreviewStartHelperTask != null && (_PreviewPlayer.Length > 0 || _PreviewStartWaitCounter++ >= _PreviewStartWaitMaxTries)) { lock (_PreviewStartHelperTaskLock) @@ -419,6 +435,27 @@ namespace Vocaluxe.Base return; } + // Song server: download the audio first, the preview starts in Update() when it's there. + // The video is downloaded in the background (the preview shows it from the next time on). + if (!string.IsNullOrEmpty(song.Video)) + { + CSongFiles.RequestAsync(new[] { song.GetVideo() }); + } + + if (!CSongFiles.IsLocal(song.GetMP3())) + { + if (IsPlayingPreview) + { + _PreviewPlayer.Stop(); + } + + _PendingPreviewSong = song; + _PendingPreviewStart = start; + _PendingPreviewTask = CSongFiles.RequestAsync(new[] { song.GetMP3() }); + return; + } + + _PendingPreviewSong = null; _PreviewPlayer.Load(song); lock (_PreviewStartHelperTaskLock) @@ -456,6 +493,7 @@ namespace Vocaluxe.Base public static void StopPreview() { + _PendingPreviewSong = null; if (!IsPlayingPreview) { return; @@ -520,7 +558,8 @@ namespace Vocaluxe.Base return; } - foreach (var song in CSongs.AllSongs) + // Songs from the song server only when they are downloaded already + foreach (var song in CSongs.AllSongs.Where(s => CSongFiles.IsLocal(s.GetMP3()))) { _NotPlayedFiles.Add(new CPlaylistElement(song)); } diff --git a/Vocaluxe/Base/CConfig.cs b/Vocaluxe/Base/CConfig.cs index 405843ce..ed60b014 100644 --- a/Vocaluxe/Base/CConfig.cs +++ b/Vocaluxe/Base/CConfig.cs @@ -249,6 +249,14 @@ namespace Vocaluxe.Base [DefaultValue(70)] public int SongCountCoverThreshold; } + + public struct SConfigSongServer + { + [DefaultValue("")] + public string Url; + [DefaultValue("")] + public string ApiKey; + } #pragma warning restore 649 // ReSharper restore UnassignedField.Global @@ -265,6 +273,7 @@ namespace Vocaluxe.Base public SConfigVideo Video; public SConfigRecord Record; public SConfigServer Server; + public SConfigSongServer SongServer; } public static SConfig Config; @@ -643,6 +652,10 @@ namespace Vocaluxe.Base case "SongCountCoverThreshold": return "Threshold of songs for that covers will not longer be included in get-all-songs-requests (e.g. song list) (default: 70) [-1..65535] -1 => always deliver covers"; + case "Url": + return "Song server (VocaluxeSongServer) address, e.g. http://192.168.1.108:8099. Empty = no song server, only local song folders"; + case "ApiKey": + return "API key of the song server, if it requires one"; case "Stretch": return "Stretch view to full window size: " + CHelper.ListStrings(Enum.GetNames(typeof(EOffOn))); default: diff --git a/Vocaluxe/Base/CMain.cs b/Vocaluxe/Base/CMain.cs index ca1ade46..e1be876d 100644 --- a/Vocaluxe/Base/CMain.cs +++ b/Vocaluxe/Base/CMain.cs @@ -19,6 +19,7 @@ using System; using System.Collections.Generic; using System.Collections.ObjectModel; using System.Drawing; +using System.Linq; using Vocaluxe.Base.Fonts; using Vocaluxe.Base.ThemeSystem; using VocaluxeLib; @@ -133,7 +134,7 @@ namespace Vocaluxe.Base public IEnumerable GetSongFolders() { - return CConfig.SongFolders; + return CConfig.SongFolders.Concat(SongServer.CSongServer.SongFolders); } public ESongSorting GetSongSorting() diff --git a/Vocaluxe/Base/CSongs.cs b/Vocaluxe/Base/CSongs.cs index 15b96f3d..9e5da3ca 100644 --- a/Vocaluxe/Base/CSongs.cs +++ b/Vocaluxe/Base/CSongs.cs @@ -23,6 +23,7 @@ using System.IO; using System.Linq; using System.Threading; using System.Threading.Tasks; +using Vocaluxe.Base.SongServer; using VocaluxeLib; using VocaluxeLib.Log; using VocaluxeLib.Songs; @@ -418,11 +419,14 @@ namespace Vocaluxe.Base IEnumerable files; + // Song server: mirror the song text files into the cache folder, which is then loaded like a song folder. + CSongServer.Sync(); + using (CBenchmark.Time("List Songs")) { var fileSet = new HashSet(StringComparer.OrdinalIgnoreCase); - foreach (var path in CConfig.SongFolders) + foreach (var path in CConfig.SongFolders.Concat(CSongServer.SongFolders)) { if (!Directory.Exists(path)) { diff --git a/Vocaluxe/Base/SongServer/CRemoteSongFileProvider.cs b/Vocaluxe/Base/SongServer/CRemoteSongFileProvider.cs new file mode 100644 index 00000000..7520fa16 --- /dev/null +++ b/Vocaluxe/Base/SongServer/CRemoteSongFileProvider.cs @@ -0,0 +1,217 @@ +#region license +// This file is part of Vocaluxe. +// +// Vocaluxe is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Vocaluxe is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with Vocaluxe. If not, see . +#endregion + +using System; +using System.Collections.Concurrent; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Net; +using System.Net.Http; +using System.Net.Http.Headers; +using System.Threading; +using System.Threading.Tasks; +using VocaluxeLib.Log; +using VocaluxeLib.Songs; +using VocaluxeSongServer.Contracts; + +namespace Vocaluxe.Base.SongServer +{ + /// + /// Song files of the song server: the cache folder mirrors the server library, files are downloaded on demand. + /// Files outside the cache folder (local song folders) are passed through to the file system. + /// + class CRemoteSongFileProvider : ISongFileProvider + { + private const int _MaxParallelDownloads = 4; + + private readonly HttpClient _Http; + private readonly string _CacheFolder; + // Full local path -> server file. Case-insensitive: song files often reference media with different case. + private readonly Dictionary _Files; + private readonly ConcurrentDictionary> _Downloads = new ConcurrentDictionary>(StringComparer.OrdinalIgnoreCase); + private readonly ConcurrentDictionary _BytesDone = new ConcurrentDictionary(StringComparer.OrdinalIgnoreCase); + private readonly SemaphoreSlim _Parallel = new SemaphoreSlim(_MaxParallelDownloads); + + public CRemoteSongFileProvider(HttpClient http, string cacheFolder, IEnumerable files) + { + _Http = http; + _CacheFolder = cacheFolder; + _Files = new Dictionary(StringComparer.OrdinalIgnoreCase); + foreach (var file in files) + { + _Files[GetLocalPath(cacheFolder, file.Path)] = file; + } + } + + public static string GetLocalPath(string cacheFolder, string relativePath) + { + return Path.GetFullPath(Path.Combine(cacheFolder, relativePath.Replace('/', Path.DirectorySeparatorChar))); + } + + private FileDto _GetRemote(string path) + { + if (string.IsNullOrEmpty(path)) + { + return null; + } + + FileDto file; + return _Files.TryGetValue(Path.GetFullPath(path), out file) ? file : null; + } + + /// True if the local copy matches the server file (same size and modification time). + public static bool IsCurrent(string localPath, FileDto file) + { + var info = new FileInfo(localPath); + if (!info.Exists || info.Length != file.Size) + { + return false; + } + + var modified = new DateTimeOffset(info.LastWriteTimeUtc).ToUnixTimeMilliseconds(); + return Math.Abs(modified - file.Modified) < 2000; + } + + public bool Exists(string path) + { + return _GetRemote(path) != null || File.Exists(path); + } + + public bool IsLocal(string path) + { + var file = _GetRemote(path); + return file == null ? File.Exists(path) : IsCurrent(_LocalPathOf(file), file); + } + + private string _LocalPathOf(FileDto file) + { + return GetLocalPath(_CacheFolder, file.Path); + } + + public Task RequestAsync(IEnumerable paths) + { + var tasks = new List(); + foreach (var path in paths) + { + var file = _GetRemote(path); + if (file == null || IsCurrent(_LocalPathOf(file), file)) + { + continue; + } + + var localPath = _LocalPathOf(file); + // Lazy: GetOrAdd may call the factory twice under contention, but only one download may start. + tasks.Add(_Downloads.GetOrAdd(localPath, _ => new Lazy(() => _DownloadAsync(file, localPath))).Value); + } + + return Task.WhenAll(tasks); + } + + public float GetProgress(IEnumerable paths) + { + long total = 0, done = 0; + foreach (var path in paths) + { + var file = _GetRemote(path); + if (file == null) + { + continue; + } + + total += file.Size; + var localPath = _LocalPathOf(file); + long bytes; + if (IsCurrent(localPath, file)) + { + done += file.Size; + } + else if (_BytesDone.TryGetValue(localPath, out bytes)) + { + done += Math.Min(bytes, file.Size); + } + } + + return total == 0 ? 1f : (float)done / total; + } + + private async Task _DownloadAsync(FileDto file, string localPath) + { + await _Parallel.WaitAsync().ConfigureAwait(false); + try + { + Directory.CreateDirectory(Path.GetDirectoryName(localPath)); + var partPath = localPath + ".part"; + long offset = File.Exists(partPath) ? new FileInfo(partPath).Length : 0; + if (offset >= file.Size) + { + offset = 0; // stale part file + } + + using (var request = new HttpRequestMessage(HttpMethod.Get, CSongServer.FileUrl(file.Path))) + { + if (offset > 0) + { + // Resume an interrupted download, but only if the file hasn't changed since. + request.Headers.Range = new RangeHeaderValue(offset, null); + request.Headers.IfRange = new RangeConditionHeaderValue(DateTimeOffset.FromUnixTimeMilliseconds(file.Modified)); + } + + using (var response = await _Http.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false)) + { + response.EnsureSuccessStatusCode(); + var append = offset > 0 && response.StatusCode == HttpStatusCode.PartialContent; + if (!append) + { + offset = 0; + } + + _BytesDone[localPath] = offset; + using (var source = await response.Content.ReadAsStreamAsync().ConfigureAwait(false)) + using (var target = new FileStream(partPath, append ? FileMode.Append : FileMode.Create, FileAccess.Write, FileShare.None, 81920, true)) + { + var buffer = new byte[81920]; + int read; + while ((read = await source.ReadAsync(buffer, 0, buffer.Length).ConfigureAwait(false)) > 0) + { + await target.WriteAsync(buffer, 0, read).ConfigureAwait(false); + _BytesDone.AddOrUpdate(localPath, read, (_, v) => v + read); + } + } + } + } + + File.Move(partPath, localPath, true); + File.SetLastWriteTimeUtc(localPath, DateTimeOffset.FromUnixTimeMilliseconds(file.Modified).UtcDateTime); + if (!IsCurrent(localPath, file)) + { + CLog.Warning("Song server file {File} has an unexpected size after download", CLog.Params(file.Path)); + } + } + catch (Exception e) + { + CLog.Error(e, "Downloading {File} from the song server failed: {ExceptionMessage}", CLog.Params(file.Path, e.Message)); + } + finally + { + _BytesDone.TryRemove(localPath, out _); + _Downloads.TryRemove(localPath, out _); + _Parallel.Release(); + } + } + } +} diff --git a/Vocaluxe/Base/SongServer/CSongServer.cs b/Vocaluxe/Base/SongServer/CSongServer.cs new file mode 100644 index 00000000..a475d2a9 --- /dev/null +++ b/Vocaluxe/Base/SongServer/CSongServer.cs @@ -0,0 +1,264 @@ +#region license +// This file is part of Vocaluxe. +// +// Vocaluxe is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Vocaluxe is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with Vocaluxe. If not, see . +#endregion + +using System; +using System.Collections.Generic; +using System.IO; +using System.IO.Compression; +using System.Linq; +using System.Net; +using System.Net.Http; +using System.Text; +using System.Text.Json; +using VocaluxeLib.Log; +using VocaluxeLib.Songs; +using VocaluxeSongServer.Contracts; + +namespace Vocaluxe.Base.SongServer +{ + /// + /// Client of the song server (VocaluxeSongServer). The song text files are mirrored into a local cache folder, + /// which is then loaded like a normal song folder. Audio, video and images are downloaded on demand by + /// . Without connection, the cached library of the last start is used. + /// + static class CSongServer + { + private const string _LibraryFileName = "library.json"; + private const int _BundleSize = 1000; + + private static readonly JsonSerializerOptions _Json = new JsonSerializerOptions(JsonSerializerDefaults.Web); + private static HttpClient _Http; + private static string _BaseUrl; + + public static bool Enabled + { + get { return !string.IsNullOrWhiteSpace(CConfig.Config.SongServer.Url); } + } + + /// The local mirror of the server library, or null if no song server is configured. + public static string CacheFolder { get; private set; } + + /// True if the last sync reached the server, false if the cached library was used. + public static bool Online { get; private set; } + + /// Song folders to load in addition to the local ones (the cache folder if a song server is configured). + public static IEnumerable SongFolders + { + get { return CacheFolder != null ? new[] { CacheFolder } : new string[0]; } + } + + public static string FileUrl(string relativePath) + { + return _BaseUrl + "/api/v1/files/" + string.Join("/", relativePath.Split('/').Select(Uri.EscapeDataString)); + } + + private static void _Init() + { + _BaseUrl = CConfig.Config.SongServer.Url.Trim().TrimEnd('/'); + if (!_BaseUrl.Contains("://")) + { + _BaseUrl = "http://" + _BaseUrl; + } + + var uri = new Uri(_BaseUrl); + var folderName = uri.Host + "_" + uri.Port; + CacheFolder = Path.GetFullPath(Path.Combine(CSettings.DataFolder, "SongServer", folderName)); + Directory.CreateDirectory(CacheFolder); + + _Http?.Dispose(); + _Http = new HttpClient(new HttpClientHandler { AutomaticDecompression = DecompressionMethods.All }) { Timeout = TimeSpan.FromMinutes(30) }; + if (!string.IsNullOrEmpty(CConfig.Config.SongServer.ApiKey)) + { + _Http.DefaultRequestHeaders.Add("X-Api-Key", CConfig.Config.SongServer.ApiKey); + } + } + + /// + /// Gets the library from the server and mirrors the song text files. Blocks; called by the song loader thread + /// before the song folders are read. Does nothing if no song server is configured. + /// + public static void Sync() + { + CacheFolder = null; + CSongFiles.Provider = new CLocalSongFileProvider(); + if (!Enabled) + { + return; + } + + try + { + _Init(); + } + catch (Exception e) + { + CLog.Error(e, "Invalid song server address {Url}: {ExceptionMessage}", CLog.Params(CConfig.Config.SongServer.Url, e.Message)); + return; + } + + using (CBenchmark.Time("Sync song server")) + { + var libraryPath = Path.Combine(CacheFolder, _LibraryFileName); + var cached = _LoadCachedLibrary(libraryPath); + var library = _FetchLibrary(cached, libraryPath) ?? cached; + if (library == null) + { + CLog.Warning("Song server {Url} not reachable and no cached library", CLog.Params(_BaseUrl)); + return; + } + + if (Online) + { + try + { + _SyncTextFiles(library); + _RemoveStaleFiles(library); + } + catch (Exception e) + { + CLog.Error(e, "Syncing song text files from {Url} failed: {ExceptionMessage}", CLog.Params(_BaseUrl, e.Message)); + } + } + + CSongFiles.Provider = new CRemoteSongFileProvider(_Http, CacheFolder, library.Files); + CLog.Information("Song server {Url}: {Songs} songs ({State})", CLog.Params(_BaseUrl, library.Songs.Count, Online ? "online" : "offline, cached library")); + } + } + + private static LibraryDto _LoadCachedLibrary(string libraryPath) + { + try + { + return File.Exists(libraryPath) ? JsonSerializer.Deserialize(File.ReadAllText(libraryPath), _Json) : null; + } + catch (Exception e) + { + CLog.Warning("Cached song server library {File} is unreadable: {ExceptionMessage}", CLog.Params(libraryPath, e.Message)); + return null; + } + } + + private static LibraryDto _FetchLibrary(LibraryDto cached, string libraryPath) + { + Online = false; + try + { + using (var request = new HttpRequestMessage(HttpMethod.Get, _BaseUrl + "/api/v1/library")) + using (var timeout = new System.Threading.CancellationTokenSource(TimeSpan.FromSeconds(15))) + { + if (cached != null) + { + request.Headers.TryAddWithoutValidation("If-None-Match", "\"" + cached.Version + "\""); + } + + using (var response = _Http.SendAsync(request, timeout.Token).Result) + { + if (response.StatusCode == HttpStatusCode.NotModified) + { + Online = true; + return cached; + } + + response.EnsureSuccessStatusCode(); + var json = response.Content.ReadAsStringAsync().Result; + var library = JsonSerializer.Deserialize(json, _Json); + File.WriteAllText(libraryPath + ".tmp", json, Encoding.UTF8); + File.Move(libraryPath + ".tmp", libraryPath, true); + Online = true; + return library; + } + } + } + catch (Exception e) + { + var inner = e is AggregateException ? e.InnerException ?? e : e; + CLog.Warning("Can't reach song server {Url}: {ExceptionMessage}", CLog.Params(_BaseUrl, inner.Message)); + return null; + } + } + + /// Downloads the song text files that are missing or changed (as zip bundles). + private static void _SyncTextFiles(LibraryDto library) + { + var files = library.Files.ToDictionary(f => f.Path, StringComparer.Ordinal); + var missing = library.Songs + .Select(s => s.Path) + .Where(p => files.ContainsKey(p) && !CRemoteSongFileProvider.IsCurrent(CRemoteSongFileProvider.GetLocalPath(CacheFolder, p), files[p])) + .ToList(); + if (missing.Count == 0) + { + return; + } + + CLog.Information("Downloading {Count} song text files from the song server", CLog.Params(missing.Count)); + for (var i = 0; i < missing.Count; i += _BundleSize) + { + var request = new BundleRequest { Paths = missing.Skip(i).Take(_BundleSize).ToList() }; + var content = new StringContent(JsonSerializer.Serialize(request, _Json), Encoding.UTF8, "application/json"); + using (var response = _Http.PostAsync(_BaseUrl + "/api/v1/files/bundle", content).Result) + { + response.EnsureSuccessStatusCode(); + using (var stream = response.Content.ReadAsStreamAsync().Result) + using (var zip = new ZipArchive(stream, ZipArchiveMode.Read)) + { + foreach (var entry in zip.Entries) + { + FileDto file; + if (!files.TryGetValue(entry.FullName, out file)) + { + continue; + } + + var localPath = CRemoteSongFileProvider.GetLocalPath(CacheFolder, file.Path); + if (!localPath.StartsWith(CacheFolder, StringComparison.OrdinalIgnoreCase)) + { + continue; // never write outside the cache folder + } + + Directory.CreateDirectory(Path.GetDirectoryName(localPath)); + entry.ExtractToFile(localPath, true); + File.SetLastWriteTimeUtc(localPath, DateTimeOffset.FromUnixTimeMilliseconds(file.Modified).UtcDateTime); + } + } + } + } + } + + /// The cache mirrors the server: files that are no longer on the server are deleted (removed songs disappear). + private static void _RemoveStaleFiles(LibraryDto library) + { + var known = new HashSet(library.Files.Select(f => CRemoteSongFileProvider.GetLocalPath(CacheFolder, f.Path)), StringComparer.OrdinalIgnoreCase); + var libraryPath = Path.Combine(CacheFolder, _LibraryFileName); + foreach (var file in Directory.EnumerateFiles(CacheFolder, "*", SearchOption.AllDirectories)) + { + if (known.Contains(file) || string.Equals(file, libraryPath, StringComparison.OrdinalIgnoreCase) || file.EndsWith(".part", StringComparison.OrdinalIgnoreCase)) + { + continue; + } + + try + { + File.Delete(file); + } + catch (Exception e) + { + CLog.Warning("Can't delete {File} from the song cache: {ExceptionMessage}", CLog.Params(file, e.Message)); + } + } + } + } +} diff --git a/Vocaluxe/Screens/CScreenSing.cs b/Vocaluxe/Screens/CScreenSing.cs index b050ca10..333ddbc5 100644 --- a/Vocaluxe/Screens/CScreenSing.cs +++ b/Vocaluxe/Screens/CScreenSing.cs @@ -255,6 +255,64 @@ namespace Vocaluxe.Screens _Texts[_TextShortInfoTop].Visible = false; } + // Song server: files of the current song being downloaded before it starts + private System.Threading.Tasks.Task _DownloadTask; + + /// + /// Loads and starts the current song. Songs from the song server are downloaded first if needed; + /// UpdateGame() shows the progress and starts the song when the download is finished. + /// + private void _LoadAndStartCurrentSong() + { + var song = CGame.IsFinished() ? null : CGame.GetSong(); + if (song != null) + { + var files = song.GetMediaFiles(CConfig.Config.Video.VideosInSongs == EOffOn.TR_CONFIG_ON); + if (!CSongFiles.AreLocal(files)) + { + _DownloadTask = CSongFiles.RequestAsync(files); + _ShowDownloadProgress(song, files); + return; + } + } + + _LoadCurrentSong(); + _StartSong(); + } + + private void _ShowDownloadProgress(CSong song, List files) + { + var percent = (int)(CSongFiles.GetProgress(files) * 100); + _ShowInfoText(CBase.Language.Translate("TR_SCREENSING_DOWNLOADING").Replace("%s", song.Artist + " - " + song.Title).Replace("%p", percent.ToString())); + } + + /// Returns true while the current song is being downloaded. + private bool _UpdateDownload() + { + if (_DownloadTask == null) + { + return false; + } + + var song = CGame.GetSong(); + if (!_DownloadTask.IsCompleted && song != null) + { + _ShowDownloadProgress(song, song.GetMediaFiles(CConfig.Config.Video.VideosInSongs == EOffOn.TR_CONFIG_ON)); + return true; + } + + _DownloadTask = null; + if (song != null && !CSongFiles.IsLocal(song.GetMP3())) + { + CLog.Error("Song files of {Artist} - {Title} could not be downloaded, skipping the song", CLog.Params(song.Artist, song.Title)); + } + + // Video, backgrounds or cover may be missing after a failed download; the song still starts without them. + _LoadCurrentSong(); + _StartSong(); + return true; + } + private void _ShowInfoText(String text) { _Texts[_TextShortInfoTop].Text = text; @@ -630,6 +688,7 @@ namespace Vocaluxe.Screens public override void OnClose() { base.OnClose(); + _DownloadTask = null; _CloseSong(); if (_Webcam) { @@ -643,6 +702,11 @@ namespace Vocaluxe.Screens #region update game public override bool UpdateGame() { + if (_UpdateDownload()) + { + return true; + } + var finish = false; if (CSound.IsPlaying(_CurrentStream) || CSound.IsPaused(_CurrentStream)) { @@ -1534,9 +1598,7 @@ namespace Vocaluxe.Screens _CloseSong(); CGame.ResetPlayer(); - _LoadCurrentSong(); - - _StartSong(); + _LoadAndStartCurrentSong(); } /// @@ -1551,9 +1613,7 @@ namespace Vocaluxe.Screens CGame.NextRound(); _SetVisibility(); - _LoadCurrentSong(); - - _StartSong(); + _LoadAndStartCurrentSong(); } /// @@ -1586,9 +1646,7 @@ namespace Vocaluxe.Screens } - _LoadCurrentSong(); - - _StartSong(); + _LoadAndStartCurrentSong(); } private void _TogglePause() diff --git a/Vocaluxe/Vocaluxe.csproj b/Vocaluxe/Vocaluxe.csproj index fb464225..d7fbdf8b 100644 --- a/Vocaluxe/Vocaluxe.csproj +++ b/Vocaluxe/Vocaluxe.csproj @@ -27,6 +27,8 @@ + + diff --git a/VocaluxeLib/Songs/CSong.cs b/VocaluxeLib/Songs/CSong.cs index 4258617e..ac255d65 100644 --- a/VocaluxeLib/Songs/CSong.cs +++ b/VocaluxeLib/Songs/CSong.cs @@ -338,6 +338,41 @@ namespace VocaluxeLib.Songs return Path.Combine(Folder, Video); } + public string GetCover() + { + return Cover == "" ? "" : Path.Combine(Folder, Cover); + } + + /// + /// Full paths of the files needed to sing the song: audio (or instrumental/vocals), video, backgrounds and cover. + /// + public List GetMediaFiles(bool includeVideo = true) + { + var files = new List { GetMP3() }; + if (HasInstrumental()) + { + files.Add(GetInstrumental()); + } + + if (HasVocals()) + { + files.Add(GetVocals()); + } + + if (includeVideo && !string.IsNullOrEmpty(Video)) + { + files.Add(GetVideo()); + } + + files.AddRange(BackgroundFileNames.Where(bg => !string.IsNullOrEmpty(bg)).Select(bg => Path.Combine(Folder, bg))); + if (Cover != "") + { + files.Add(GetCover()); + } + + return files; + } + public void LoadSmallCover() { if (_CoverTextureSmall != null) @@ -347,7 +382,9 @@ namespace VocaluxeLib.Songs if (Cover != "") { - if (CBase.DataBase.GetCover(Path.Combine(Folder, Cover), ref _CoverTextureSmall, CBase.Config.GetCoverSize())) + // Songs from a song server: download the cover first (cover loading runs on background threads). + CSongFiles.EnsureLocal(GetCover()); + if (CBase.DataBase.GetCover(GetCover(), ref _CoverTextureSmall, CBase.Config.GetCoverSize())) { return; } diff --git a/VocaluxeLib/Songs/CSongFiles.cs b/VocaluxeLib/Songs/CSongFiles.cs new file mode 100644 index 00000000..1d23b82d --- /dev/null +++ b/VocaluxeLib/Songs/CSongFiles.cs @@ -0,0 +1,125 @@ +#region license +// This file is part of Vocaluxe. +// +// Vocaluxe is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Vocaluxe is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with Vocaluxe. If not, see . +#endregion + +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Threading.Tasks; + +namespace VocaluxeLib.Songs +{ + /// + /// Access to the files of a song (audio, video, cover, backgrounds). + /// Songs from a song server exist locally only as their text file at first; their other files are + /// downloaded on demand. Everything that opens a song file should check and + /// call (or on background threads) first. + /// + public interface ISongFileProvider + { + /// True if the file exists locally or can be downloaded. + bool Exists(string path); + + /// True if the file can be opened now (exists locally and is complete). + bool IsLocal(string path); + + /// Downloads the files that are not local yet. The task completes when all are local (or failed). + Task RequestAsync(IEnumerable paths); + + /// Progress of the downloads of these files in [0..1], 1 if all are local. + float GetProgress(IEnumerable paths); + } + + /// The default provider: plain local files. + public class CLocalSongFileProvider : ISongFileProvider + { + public bool Exists(string path) + { + return File.Exists(path); + } + + public bool IsLocal(string path) + { + return File.Exists(path); + } + + public Task RequestAsync(IEnumerable paths) + { + return Task.CompletedTask; + } + + public float GetProgress(IEnumerable paths) + { + return 1f; + } + } + + public static class CSongFiles + { + public static ISongFileProvider Provider { get; set; } = new CLocalSongFileProvider(); + + public static bool Exists(string path) + { + return !string.IsNullOrEmpty(path) && Provider.Exists(path); + } + + public static bool IsLocal(string path) + { + return !string.IsNullOrEmpty(path) && Provider.IsLocal(path); + } + + public static bool AreLocal(IEnumerable paths) + { + return paths.All(IsLocal); + } + + public static Task RequestAsync(IEnumerable paths) + { + var missing = paths.Where(p => !string.IsNullOrEmpty(p) && !Provider.IsLocal(p)).Distinct().ToList(); + return missing.Count == 0 ? Task.CompletedTask : Provider.RequestAsync(missing); + } + + /// Blocks until the file is local. Only for background threads (e.g. cover loading). + public static bool EnsureLocal(string path) + { + if (IsLocal(path)) + { + return true; + } + + if (!Exists(path)) + { + return false; + } + + try + { + RequestAsync(new[] { path }).Wait(); + } + catch + { + // A failed download is reported by the provider; the caller handles the missing file. + } + + return IsLocal(path); + } + + public static float GetProgress(IEnumerable paths) + { + return Provider.GetProgress(paths.Where(p => !string.IsNullOrEmpty(p))); + } + } +} diff --git a/VocaluxeLib/Songs/CSongLoader.cs b/VocaluxeLib/Songs/CSongLoader.cs index 2735e823..c4e21876 100644 --- a/VocaluxeLib/Songs/CSongLoader.cs +++ b/VocaluxeLib/Songs/CSongLoader.cs @@ -194,7 +194,7 @@ namespace VocaluxeLib.Songs break; case "MP3": case "AUDIO": - if (File.Exists(Path.Combine(_Song.Folder, value))) + if (CSongFiles.Exists(Path.Combine(_Song.Folder, value))) { _Song.Audio = value; headerFlags |= EHeaderFlags.MP3; @@ -207,7 +207,7 @@ namespace VocaluxeLib.Songs break; case "INSTRUMENTAL": - if (File.Exists(Path.Combine(_Song.Folder, value))) + if (CSongFiles.Exists(Path.Combine(_Song.Folder, value))) { _Song.Instrumental = value; headerFlags |= EHeaderFlags.Instrumental; @@ -221,7 +221,7 @@ namespace VocaluxeLib.Songs break; case "VOCALS": - if (File.Exists(Path.Combine(_Song.Folder, value))) + if (CSongFiles.Exists(Path.Combine(_Song.Folder, value))) { _Song.Vocals = value; headerFlags |= EHeaderFlags.Vocals; @@ -298,7 +298,7 @@ namespace VocaluxeLib.Songs break; case "COVER": - if (File.Exists(Path.Combine(_Song.Folder, value))) + if (CSongFiles.Exists(Path.Combine(_Song.Folder, value))) { _Song.Cover = value; } @@ -309,7 +309,7 @@ namespace VocaluxeLib.Songs break; case "BACKGROUND": - if (File.Exists(Path.Combine(_Song.Folder, value))) + if (CSongFiles.Exists(Path.Combine(_Song.Folder, value))) { _Song.BackgroundFileNames.Add(value); } @@ -321,7 +321,7 @@ namespace VocaluxeLib.Songs break; case "VIDEO": - if (File.Exists(Path.Combine(_Song.Folder, value))) + if (CSongFiles.Exists(Path.Combine(_Song.Folder, value))) { _Song.Video = value; } diff --git a/VocaluxeLib/Utils/Player/CSongPlayer.cs b/VocaluxeLib/Utils/Player/CSongPlayer.cs index 9b758d3d..6fc666c7 100644 --- a/VocaluxeLib/Utils/Player/CSongPlayer.cs +++ b/VocaluxeLib/Utils/Player/CSongPlayer.cs @@ -76,7 +76,7 @@ namespace VocaluxeLib.Utils.Player { var song = _Song; return song != null && !string.IsNullOrEmpty(song.Folder) && !string.IsNullOrEmpty(song.Video) && - File.Exists(Path.Combine(song.Folder, song.Video)); + CSongFiles.IsLocal(song.GetVideo()); } } diff --git a/VocaluxeSongServer/Contracts/LibraryDtos.cs b/VocaluxeSongServer/Contracts/LibraryDtos.cs new file mode 100644 index 00000000..0a3f4cad --- /dev/null +++ b/VocaluxeSongServer/Contracts/LibraryDtos.cs @@ -0,0 +1,58 @@ +#region license +// This file is part of Vocaluxe. +// +// Vocaluxe is free software: you can redistribute it and/or modify +// it under the terms of the GNU General Public License as published by +// the Free Software Foundation, either version 3 of the License, or +// (at your option) any later version. +// +// Vocaluxe is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +// GNU General Public License for more details. +// +// You should have received a copy of the GNU General Public License +// along with Vocaluxe. If not, see . +#endregion + +using System.Collections.Generic; + +// Shared between the song server and the game (linked into Vocaluxe.csproj). Keep it free of dependencies. +// Serialized as camelCase JSON. All paths are relative to the library root and use '/'. +namespace VocaluxeSongServer.Contracts +{ + /// GET /api/v1/library + public sealed class LibraryDto + { + /// Changes whenever any song file is added, removed or changed. Also sent as ETag. + public string Version { get; set; } = ""; + public string ScannedAt { get; set; } = ""; + public List Songs { get; set; } = new List(); + /// All files in the song folders (text files, audio, video, images), downloadable via /api/v1/files/{path}. + public List Files { get; set; } = new List(); + } + + public sealed class SongDto + { + /// Stable id, derived from the path of the text file. + public string Id { get; set; } = ""; + /// Path of the UltraStar text file. + public string Path { get; set; } = ""; + public string Title { get; set; } = ""; + public string Artist { get; set; } = ""; + } + + public sealed class FileDto + { + public string Path { get; set; } = ""; + public long Size { get; set; } + /// Last modification, Unix time in milliseconds (UTC). + public long Modified { get; set; } + } + + /// POST /api/v1/files/bundle: returns the files as a zip archive (entries named by their path). + public sealed class BundleRequest + { + public List Paths { get; set; } = new List(); + } +} diff --git a/VocaluxeSongServer/Dockerfile b/VocaluxeSongServer/Dockerfile new file mode 100644 index 00000000..0990ab11 --- /dev/null +++ b/VocaluxeSongServer/Dockerfile @@ -0,0 +1,18 @@ +# Build from the repository root: docker build -f VocaluxeSongServer/Dockerfile -t vocaluxe-song-server . +FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build +WORKDIR /src +COPY Directory.Build.props NuGet.Config ./ +COPY VocaluxeSongServer/VocaluxeSongServer.csproj VocaluxeSongServer/ +RUN dotnet restore VocaluxeSongServer/VocaluxeSongServer.csproj +COPY VocaluxeSongServer/ VocaluxeSongServer/ +RUN dotnet publish VocaluxeSongServer/VocaluxeSongServer.csproj -c ReleaseLinux --no-restore -o /app + +FROM mcr.microsoft.com/dotnet/aspnet:10.0 +WORKDIR /app +COPY --from=build /app . +# The song library (read-only is enough); see README.md +VOLUME /songs +ENV Library__Path=/songs +EXPOSE 8080 +USER $APP_UID +ENTRYPOINT ["dotnet", "VocaluxeSongServer.dll"] diff --git a/VocaluxeSongServer/Library/CLibraryIndex.cs b/VocaluxeSongServer/Library/CLibraryIndex.cs new file mode 100644 index 00000000..ce0424a7 --- /dev/null +++ b/VocaluxeSongServer/Library/CLibraryIndex.cs @@ -0,0 +1,106 @@ +using Microsoft.Extensions.Options; + +namespace VocaluxeSongServer.Library +{ + public sealed class CLibraryOptions + { + /// Root folder of the song library (in Docker: the mounted share, default /songs). + public string Path { get; set; } = "/songs"; + /// Minutes between automatic rescans. 0 = only at startup and via POST /api/v1/library/rescan. + public int ScanIntervalMinutes { get; set; } = 10; + /// If set, every /api request needs the header X-Api-Key with this value. + public string ApiKey { get; set; } = ""; + } + + /// Holds the current library snapshot and rescans it in the background. + public sealed class CLibraryIndex : BackgroundService + { + private readonly CLibraryScanner _Scanner; + private readonly CLibraryOptions _Options; + private readonly ILogger _Logger; + private readonly SemaphoreSlim _ScanLock = new(1, 1); + private readonly SemaphoreSlim _RescanRequested = new(0, 1); + private volatile CLibrarySnapshot? _Current; + + public CLibraryIndex(CLibraryScanner scanner, IOptions options, ILogger logger) + { + _Scanner = scanner; + _Options = options.Value; + _Logger = logger; + } + + /// The last completed scan, or null while the first scan is running. + public CLibrarySnapshot? Current => _Current; + + public string Root => System.IO.Path.GetFullPath(_Options.Path); + + /// Asks for a rescan as soon as possible (returns immediately). + public void RequestRescan() + { + if (_RescanRequested.CurrentCount == 0) + { + try + { + _RescanRequested.Release(); + } + catch (SemaphoreFullException) + { + // Already requested. + } + } + } + + public async Task ScanNowAsync(CancellationToken ct = default) + { + await _ScanLock.WaitAsync(ct); + try + { + if (!Directory.Exists(Root)) + { + _Logger.LogError("Library folder {Root} does not exist", Root); + return; + } + + var started = DateTime.UtcNow; + var snapshot = await Task.Run(() => _Scanner.Scan(Root), ct); + var changed = _Current?.Dto.Version != snapshot.Dto.Version; + _Current = snapshot; + _Logger.LogInformation("Scanned {Root}: {Songs} songs, {Files} files in {Seconds:0.0} s{Changed}", Root, + snapshot.Dto.Songs.Count, snapshot.Dto.Files.Count, (DateTime.UtcNow - started).TotalSeconds, changed ? "" : " (unchanged)"); + } + finally + { + _ScanLock.Release(); + } + } + + protected override async Task ExecuteAsync(CancellationToken stoppingToken) + { + while (!stoppingToken.IsCancellationRequested) + { + try + { + await ScanNowAsync(stoppingToken); + } + catch (OperationCanceledException) when (stoppingToken.IsCancellationRequested) + { + return; + } + catch (Exception e) + { + _Logger.LogError(e, "Library scan failed"); + } + + var interval = _Options.ScanIntervalMinutes > 0 ? TimeSpan.FromMinutes(_Options.ScanIntervalMinutes) : Timeout.InfiniteTimeSpan; + try + { + await _RescanRequested.WaitAsync(interval, stoppingToken); + } + catch (OperationCanceledException) + { + return; + } + } + } + } +} diff --git a/VocaluxeSongServer/Library/CLibraryScanner.cs b/VocaluxeSongServer/Library/CLibraryScanner.cs new file mode 100644 index 00000000..3a68f740 --- /dev/null +++ b/VocaluxeSongServer/Library/CLibraryScanner.cs @@ -0,0 +1,232 @@ +using System.Security.Cryptography; +using System.Text; +using VocaluxeSongServer.Contracts; + +namespace VocaluxeSongServer.Library +{ + /// A scanned library: the DTO sent to clients plus the lookup of downloadable files. + public sealed class CLibrarySnapshot + { + public required LibraryDto Dto { get; init; } + /// Relative path ('/') -> full path, case-sensitive like the file system on the server. + public required IReadOnlyDictionary FullPaths { get; init; } + } + + /// + /// Finds UltraStar songs (text files with #TITLE, #ARTIST and #MP3/#AUDIO) below the library root. + /// Only the header of each text file is read; unchanged files are not read again on the next scan. + /// + public sealed class CLibraryScanner + { + private sealed record CTxtInfo(long Size, long Modified, CSongHeader? Header); + + public sealed record CSongHeader(string Title, string Artist); + + private readonly ILogger _Logger; + private Dictionary _TxtCache = new(StringComparer.Ordinal); + + public CLibraryScanner(ILogger logger) + { + _Logger = logger; + } + + private static readonly EnumerationOptions _Recursive = new() + { + RecurseSubdirectories = true, + IgnoreInaccessible = true, + MatchCasing = MatchCasing.CaseInsensitive, + AttributesToSkip = FileAttributes.Hidden | FileAttributes.System + }; + + private static readonly EnumerationOptions _TopLevel = new() + { + RecurseSubdirectories = false, + IgnoreInaccessible = true, + AttributesToSkip = FileAttributes.Hidden | FileAttributes.System | FileAttributes.Directory + }; + + public CLibrarySnapshot Scan(string root) + { + root = Path.GetFullPath(root); + var songs = new List(); + var songFolders = new HashSet(StringComparer.Ordinal); + var newCache = new Dictionary(StringComparer.Ordinal); + + foreach (var txt in Directory.EnumerateFiles(root, "*.txt", _Recursive)) + { + FileInfo fi; + try + { + fi = new FileInfo(txt); + } + catch (Exception e) + { + _Logger.LogWarning("Can't read {File}: {Message}", txt, e.Message); + continue; + } + + var modified = new DateTimeOffset(fi.LastWriteTimeUtc).ToUnixTimeMilliseconds(); + if (!_TxtCache.TryGetValue(txt, out var info) || info.Size != fi.Length || info.Modified != modified) + { + info = new CTxtInfo(fi.Length, modified, ReadHeader(txt)); + } + + newCache[txt] = info; + if (info.Header == null) + { + continue; + } + + var rel = ToRelative(root, txt); + songs.Add(new SongDto { Id = SongId(rel), Path = rel, Title = info.Header.Title, Artist = info.Header.Artist }); + songFolders.Add(fi.DirectoryName!); + } + + _TxtCache = newCache; + + var files = new List(); + var fullPaths = new Dictionary(StringComparer.Ordinal); + foreach (var folder in songFolders) + { + foreach (var file in Directory.EnumerateFiles(folder, "*", _TopLevel)) + { + var fi = new FileInfo(file); + var rel = ToRelative(root, file); + files.Add(new FileDto { Path = rel, Size = fi.Length, Modified = new DateTimeOffset(fi.LastWriteTimeUtc).ToUnixTimeMilliseconds() }); + fullPaths[rel] = file; + } + } + + songs.Sort((a, b) => string.CompareOrdinal(a.Path, b.Path)); + files.Sort((a, b) => string.CompareOrdinal(a.Path, b.Path)); + + var dto = new LibraryDto + { + Version = ComputeVersion(files), + ScannedAt = DateTimeOffset.UtcNow.ToString("O"), + Songs = songs, + Files = files + }; + return new CLibrarySnapshot { Dto = dto, FullPaths = fullPaths }; + } + + public static string ToRelative(string root, string path) + { + return Path.GetRelativePath(root, path).Replace('\\', '/'); + } + + public static string SongId(string relativePath) + { + var hash = SHA256.HashData(Encoding.UTF8.GetBytes(relativePath)); + return Convert.ToHexString(hash, 0, 8).ToLowerInvariant(); + } + + private static string ComputeVersion(List files) + { + using var sha = IncrementalHash.CreateHash(HashAlgorithmName.SHA256); + foreach (var f in files) + { + sha.AppendData(Encoding.UTF8.GetBytes(f.Path + "\n" + f.Size + "\n" + f.Modified + "\n")); + } + + return Convert.ToHexString(sha.GetHashAndReset(), 0, 12).ToLowerInvariant(); + } + + /// + /// Reads the header of an UltraStar text file. Returns null if it is not a song + /// (no #TITLE, #ARTIST or #MP3/#AUDIO tag before the first note line). + /// + public CSongHeader? ReadHeader(string path) + { + try + { + // The header is at the top; 64 KB is plenty even with long comments. + byte[] bytes; + using (var fs = File.OpenRead(path)) + { + var len = (int)Math.Min(fs.Length, 64 * 1024); + bytes = new byte[len]; + fs.ReadExactly(bytes, 0, len); + if (fs.Length > len) + { + // Cut at the last line break so a UTF-8 character isn't split (which would look like Latin-1). + var lastBreak = Array.LastIndexOf(bytes, (byte)'\n'); + if (lastBreak > 0) + { + Array.Resize(ref bytes, lastBreak); + } + } + } + + return ParseHeader(DecodeText(bytes)); + } + catch (Exception e) + { + _Logger.LogWarning("Can't read {File}: {Message}", path, e.Message); + return null; + } + } + + public static CSongHeader? ParseHeader(string text) + { + string? title = null, artist = null; + var hasAudio = false; + foreach (var rawLine in text.Split('\n')) + { + var line = rawLine.Trim(); + if (line.Length == 0) + { + continue; + } + + if (line[0] != '#') + { + break; + } + + var colon = line.IndexOf(':'); + if (colon < 0) + { + continue; + } + + var tag = line.Substring(1, colon - 1).Trim().ToUpperInvariant(); + var value = line.Substring(colon + 1).Trim(); + switch (tag) + { + case "TITLE": + title = value; + break; + case "ARTIST": + artist = value; + break; + case "MP3": + case "AUDIO": + hasAudio |= value.Length > 0; + break; + } + } + + return !string.IsNullOrEmpty(title) && !string.IsNullOrEmpty(artist) && hasAudio ? new CSongHeader(title, artist) : null; + } + + /// UTF-8 (with or without BOM) if valid, otherwise Windows-1252/Latin-1 like most old song files. + public static string DecodeText(byte[] bytes) + { + var span = bytes.AsSpan(); + if (span.StartsWith(new byte[] { 0xEF, 0xBB, 0xBF })) + { + span = span[3..]; + } + + try + { + return new UTF8Encoding(false, true).GetString(span); + } + catch (DecoderFallbackException) + { + return Encoding.Latin1.GetString(span); + } + } + } +} diff --git a/VocaluxeSongServer/Program.cs b/VocaluxeSongServer/Program.cs new file mode 100644 index 00000000..e561bf30 --- /dev/null +++ b/VocaluxeSongServer/Program.cs @@ -0,0 +1,126 @@ +using System.IO.Compression; +using System.Text.Json; +using Microsoft.AspNetCore.ResponseCompression; +using Microsoft.AspNetCore.StaticFiles; +using Microsoft.Extensions.Options; +using VocaluxeSongServer.Contracts; +using VocaluxeSongServer.Library; + +var builder = WebApplication.CreateBuilder(args); + +// Settings: appsettings.json section "Library", or environment variables Library__Path, Library__ApiKey, ... +builder.Services.Configure(builder.Configuration.GetSection("Library")); +builder.Services.AddSingleton(); +builder.Services.AddSingleton(); +builder.Services.AddHostedService(sp => sp.GetRequiredService()); +builder.Services.AddResponseCompression(o => +{ + o.EnableForHttps = true; + o.Providers.Add(); + o.Providers.Add(); +}); +builder.Services.ConfigureHttpJsonOptions(o => o.SerializerOptions.PropertyNamingPolicy = JsonNamingPolicy.CamelCase); + +var app = builder.Build(); +app.UseResponseCompression(); + +app.MapGet("/health", (CLibraryIndex index) => index.Current == null + ? Results.Json(new { status = "scanning" }, statusCode: 503) + : Results.Json(new { status = "ok", songs = index.Current.Dto.Songs.Count, version = index.Current.Dto.Version })); + +var api = app.MapGroup("/api/v1"); +api.AddEndpointFilter(async (context, next) => +{ + var key = context.HttpContext.RequestServices.GetRequiredService>().Value.ApiKey; + if (!string.IsNullOrEmpty(key) && context.HttpContext.Request.Headers["X-Api-Key"] != key) + { + return Results.Problem(title: "Missing or wrong X-Api-Key", statusCode: StatusCodes.Status401Unauthorized); + } + + return await next(context); +}); + +// The whole library: songs and their files. Clients cache it and send If-None-Match. +api.MapGet("/library", (HttpContext ctx, CLibraryIndex index) => +{ + var snapshot = index.Current; + if (snapshot == null) + { + return Results.Problem(title: "The library is being scanned, try again shortly", statusCode: StatusCodes.Status503ServiceUnavailable); + } + + var etag = "\"" + snapshot.Dto.Version + "\""; + ctx.Response.Headers.ETag = etag; + if (ctx.Request.Headers.IfNoneMatch == etag) + { + return Results.StatusCode(StatusCodes.Status304NotModified); + } + + return Results.Json(snapshot.Dto); +}); + +api.MapPost("/library/rescan", (CLibraryIndex index) => +{ + index.RequestRescan(); + return Results.Accepted(); +}); + +// One file of the library. Only files listed in the library are served (no path tricks possible). +// Supports Range requests, so clients can resume downloads. +var contentTypes = new FileExtensionContentTypeProvider(); +api.MapGet("/files/{**path}", (string path, CLibraryIndex index) => +{ + var snapshot = index.Current; + if (snapshot == null || !snapshot.FullPaths.TryGetValue(path, out var fullPath) || !File.Exists(fullPath)) + { + return Results.NotFound(); + } + + if (!contentTypes.TryGetContentType(fullPath, out var contentType)) + { + contentType = "application/octet-stream"; + } + + return Results.File(fullPath, contentType, lastModified: File.GetLastWriteTimeUtc(fullPath), enableRangeProcessing: true); +}); + +// Several (small) files at once as a zip archive, e.g. all song text files on the first sync. +api.MapPost("/files/bundle", async (BundleRequest request, CLibraryIndex index, HttpContext ctx) => +{ + var snapshot = index.Current; + if (snapshot == null) + { + return Results.Problem(title: "The library is being scanned, try again shortly", statusCode: StatusCodes.Status503ServiceUnavailable); + } + + const int maxFiles = 2000; + if (request.Paths.Count > maxFiles) + { + return Results.Problem(title: $"At most {maxFiles} files per bundle", statusCode: StatusCodes.Status400BadRequest); + } + + var buffer = new MemoryStream(); + using (var zip = new ZipArchive(buffer, ZipArchiveMode.Create, true)) + { + foreach (var path in request.Paths.Distinct()) + { + if (!snapshot.FullPaths.TryGetValue(path, out var fullPath) || !File.Exists(fullPath)) + { + continue; + } + + var entry = zip.CreateEntry(path, CompressionLevel.Fastest); + entry.LastWriteTime = File.GetLastWriteTimeUtc(fullPath); + await using var source = File.OpenRead(fullPath); + await using var target = entry.Open(); + await source.CopyToAsync(target, ctx.RequestAborted); + } + } + + buffer.Position = 0; + return Results.File(buffer, "application/zip", "bundle.zip"); +}); + +app.Run(); + +public partial class Program { } diff --git a/VocaluxeSongServer/README.md b/VocaluxeSongServer/README.md new file mode 100644 index 00000000..7299ca80 --- /dev/null +++ b/VocaluxeSongServer/README.md @@ -0,0 +1,63 @@ +# Vocaluxe song server + +A central song library for Vocaluxe. The server hosts the UltraStar song folders (text file, audio, video, cover, +backgrounds). Vocaluxe mirrors the song text files at startup and downloads audio, video and images only when they are +needed, then keeps them in a local cache. + +## Running it (Docker) + +``` +docker run -d --name vocaluxe-song-server -p 8099:8080 \ + -v /mnt/user/karaoke/songs:/songs:ro \ + git.disi.dev/projects/vocaluxe-song-server:latest +``` + +| Setting (environment variable) | Default | Meaning | +|---------------------------------|----------|----------------------------------------------------------------------| +| `Library__Path` | `/songs` | Root folder of the song library; any folder structure below it works | +| `Library__ScanIntervalMinutes` | `10` | Rescan interval; `0` = only at startup and via the rescan endpoint | +| `Library__ApiKey` | empty | If set, clients must send it as `X-Api-Key` | + +A song is any `.txt` file with `#TITLE`, `#ARTIST` and `#MP3` (or `#AUDIO`). All files in its folder are served. +Files outside song folders are never served. + +## Using it in Vocaluxe + +In `config.xml`: + +```xml + + http://192.168.1.108:8099 + + +``` + +Local song folders keep working alongside the server. The cache is in the Vocaluxe data folder under +`SongServer/_`; it mirrors the server, so songs removed on the server disappear from it. Without a +connection Vocaluxe uses the library of the last start, and every song whose files were downloaded before can be sung. + +What is downloaded when: +* **Startup:** the song list and all song text files (only changed ones after the first time). Covers are downloaded by + the cover loader in the background. +* **Song selected (preview):** the audio file; the preview starts when it's there. The video is downloaded in the + background. +* **Song started:** everything still missing (video, instrumental/vocals, backgrounds). The sing screen shows the + progress and starts the song when the download is finished. + +## API + +All endpoints are under `/api/v1` (plus `GET /health`). Paths are relative to the library root and use `/`. +The types are in `Contracts/LibraryDtos.cs`, which Vocaluxe links in. + +| Endpoint | What it does | +|------------------------------|-------------------------------------------------------------------------------------| +| `GET /library` | Songs (id, path, title, artist) and all files (path, size, modified); ETag/304 | +| `GET /files/{path}` | One file, with Range support (resumable downloads) | +| `POST /files/bundle` | Several files as a zip, body `{"paths": [...]}` (max. 2000), used for text files | +| `POST /library/rescan` | Rescan now instead of waiting for the interval | + +## Building and releasing + +`dotnet run --project VocaluxeSongServer -- --Library:Path=D:\Songs --urls http://localhost:8099` runs it locally. +The image is built by `.gitea/workflows/song-server.yml`: set `VERSION`, then push the tag `songserver-v` +(pushes `:` and `:latest`). Tests: `Tests/SongServer`. diff --git a/VocaluxeSongServer/VERSION b/VocaluxeSongServer/VERSION new file mode 100644 index 00000000..6e8bf73a --- /dev/null +++ b/VocaluxeSongServer/VERSION @@ -0,0 +1 @@ +0.1.0 diff --git a/VocaluxeSongServer/VocaluxeSongServer.csproj b/VocaluxeSongServer/VocaluxeSongServer.csproj new file mode 100644 index 00000000..dfc40999 --- /dev/null +++ b/VocaluxeSongServer/VocaluxeSongServer.csproj @@ -0,0 +1,19 @@ + + + + + VocaluxeSongServer + VocaluxeSongServer + AnyCPU + enable + enable + true + + + + + + + + diff --git a/VocaluxeSongServer/appsettings.json b/VocaluxeSongServer/appsettings.json new file mode 100644 index 00000000..d3fe3e5e --- /dev/null +++ b/VocaluxeSongServer/appsettings.json @@ -0,0 +1,14 @@ +{ + "Logging": { + "LogLevel": { + "Default": "Information", + "Microsoft.AspNetCore": "Warning" + } + }, + "Urls": "http://0.0.0.0:8080", + "Library": { + "Path": "/songs", + "ScanIntervalMinutes": 10, + "ApiKey": "" + } +}