137 lines
4.5 KiB
Markdown
137 lines
4.5 KiB
Markdown
|
|
# Explicit Package Dependencies - Complete ✅
|
||
|
|
|
||
|
|
## Summary
|
||
|
|
|
||
|
|
All Python packages from both virtual environments (`.venv-cuda` and `.venv`) have been explicitly documented and will be installed in their respective Docker containers with exact version pinning.
|
||
|
|
|
||
|
|
## What Was Done
|
||
|
|
|
||
|
|
### 1. Extracted Package Lists
|
||
|
|
- Exported complete package list from `.venv-cuda` (CUDA/Python 3.11)
|
||
|
|
- Exported complete package list from `.venv` (ROCm/Python 3.10)
|
||
|
|
|
||
|
|
### 2. Created Requirements Files
|
||
|
|
|
||
|
|
**`requirements-soprano.txt`** (35 packages + PyTorch)
|
||
|
|
- Core dependencies: fastapi, uvicorn, pyzmq, numpy, pydantic
|
||
|
|
- Audio: sounddevice, pydub
|
||
|
|
- ML: lmdeploy, transformers, tokenizers, huggingface-hub, safetensors
|
||
|
|
- Supporting: accelerate, sentencepiece, protobuf, tiktoken, requests, tqdm, PyYAML, etc.
|
||
|
|
- Additional: einops, peft, scipy
|
||
|
|
|
||
|
|
**`requirements-rvc.txt`** (60+ packages)
|
||
|
|
- Core dependencies: fastapi, uvicorn, pyzmq, numpy, pydantic
|
||
|
|
- Audio processing: librosa, soundfile, sounddevice, pydub, audioread, resampy, soxr, pyworld, praat-parselmouth, torchcrepe, torchfcpe
|
||
|
|
- RVC core: fairseq, faiss-cpu, gradio, gradio_client
|
||
|
|
- ML: lmdeploy, transformers, tokenizers, huggingface-hub, safetensors
|
||
|
|
- Scientific: scipy, scikit-learn, matplotlib, pandas
|
||
|
|
- Performance: numba, llvmlite
|
||
|
|
- Additional: av, pillow, omegaconf, hydra-core, python-dotenv, ray, local-attention
|
||
|
|
|
||
|
|
### 3. Updated Dockerfiles
|
||
|
|
|
||
|
|
**`Dockerfile.soprano`** changes:
|
||
|
|
- Added `COPY requirements-soprano.txt /app/`
|
||
|
|
- Explicitly install PyTorch with CUDA 11.8: `torch==2.7.1+cu118`, `torchvision==0.22.1+cu118`, `torchaudio==2.7.1+cu118`
|
||
|
|
- Install requirements: `pip install -r requirements-soprano.txt`
|
||
|
|
- Install soprano from source: `pip install -e '.[lmdeploy]'`
|
||
|
|
|
||
|
|
**`Dockerfile.rvc`** changes:
|
||
|
|
- Added `curl` to system dependencies (for healthcheck)
|
||
|
|
- Added `COPY requirements-rvc.txt /app/`
|
||
|
|
- Added `COPY models/ /app/models/`
|
||
|
|
- Install requirements: `pip install -r requirements-rvc.txt`
|
||
|
|
- Removed old dual-install approach (requirements.txt + pyzmq separately)
|
||
|
|
|
||
|
|
### 4. Created Documentation
|
||
|
|
|
||
|
|
**`DOCKER_DEPENDENCIES.md`**
|
||
|
|
- Complete list of all packages in each container
|
||
|
|
- Version numbers explicitly listed
|
||
|
|
- Key differences between containers highlighted
|
||
|
|
- Installation order documented
|
||
|
|
- Verification commands provided
|
||
|
|
- Maintenance procedures outlined
|
||
|
|
|
||
|
|
## Key Version Pins
|
||
|
|
|
||
|
|
### Soprano Container (CUDA)
|
||
|
|
```
|
||
|
|
Python: 3.11.14
|
||
|
|
pip: 25.3
|
||
|
|
PyTorch: 2.7.1+cu118
|
||
|
|
NumPy: 2.4.1
|
||
|
|
FastAPI: 0.128.0
|
||
|
|
LMDeploy: 0.11.1
|
||
|
|
Transformers: 4.57.5
|
||
|
|
Soprano TTS: 0.1.0 (editable install)
|
||
|
|
```
|
||
|
|
|
||
|
|
### RVC Container (ROCm)
|
||
|
|
```
|
||
|
|
Python: 3.10.19 (target, from base image)
|
||
|
|
pip: 24.0
|
||
|
|
PyTorch: 2.5.1+rocm6.2 (from base image)
|
||
|
|
NumPy: 1.23.5
|
||
|
|
FastAPI: 0.128.0
|
||
|
|
LMDeploy: 0.11.1
|
||
|
|
Transformers: 4.57.3
|
||
|
|
Fairseq: 0.12.2
|
||
|
|
Librosa: 0.10.2
|
||
|
|
```
|
||
|
|
|
||
|
|
## Critical Differences
|
||
|
|
|
||
|
|
1. **Python Version**: 3.11.14 vs 3.10.19
|
||
|
|
2. **pip Version**: 25.3 vs 24.0
|
||
|
|
3. **PyTorch Version**: 2.7.1 (CUDA) vs 2.5.1 (ROCm)
|
||
|
|
4. **NumPy Version**: 2.4.1 vs 1.23.5 (RVC requires older numpy)
|
||
|
|
5. **Package Count**: ~35 (Soprano) vs ~60 (RVC)
|
||
|
|
|
||
|
|
## Benefits
|
||
|
|
|
||
|
|
✅ **No Ambiguity**: Every package version explicitly specified
|
||
|
|
✅ **Reproducible Builds**: Exact versions match working venvs
|
||
|
|
✅ **Isolated Dependencies**: Each container has only what it needs
|
||
|
|
✅ **Version Pinning**: No unexpected updates breaking compatibility
|
||
|
|
✅ **Documentation**: Complete package manifest for troubleshooting
|
||
|
|
✅ **Maintenance**: Clear reference for future updates
|
||
|
|
|
||
|
|
## Files Modified
|
||
|
|
|
||
|
|
```
|
||
|
|
soprano_to_rvc/
|
||
|
|
├── requirements-soprano.txt ✅ CREATED (35 packages)
|
||
|
|
├── requirements-rvc.txt ✅ CREATED (60+ packages)
|
||
|
|
├── Dockerfile.soprano ✅ UPDATED (explicit installs)
|
||
|
|
├── Dockerfile.rvc ✅ UPDATED (explicit installs)
|
||
|
|
└── DOCKER_DEPENDENCIES.md ✅ CREATED (documentation)
|
||
|
|
```
|
||
|
|
|
||
|
|
## Next Steps
|
||
|
|
|
||
|
|
1. **Build containers**: `./build_docker.sh`
|
||
|
|
2. **Verify packages**: Check with verification commands in DOCKER_DEPENDENCIES.md
|
||
|
|
3. **Test functionality**: Ensure performance matches bare metal
|
||
|
|
4. **Deploy**: `./start_docker.sh`
|
||
|
|
|
||
|
|
## Verification Commands
|
||
|
|
|
||
|
|
After building, verify installations match expectations:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Soprano container
|
||
|
|
docker exec miku-soprano-tts pip list | grep -E "torch|soprano|lmdeploy|numpy"
|
||
|
|
|
||
|
|
# RVC container
|
||
|
|
docker exec miku-rvc-api pip list | grep -E "torch|fairseq|librosa|numpy"
|
||
|
|
```
|
||
|
|
|
||
|
|
Expected output should match versions in requirements files.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
**Status**: All packages explicitly pinned and documented! 🎯
|
||
|
|
|
||
|
|
The Docker containers will now install exact package versions from working virtual environments with no ambiguity.
|