MP3 to MIDI: The Honest Field Guide Nobody Asked For

The short answer

MP3-to-MIDI is not a format conversion. It is audio transcription — software listens to a recorded performance and guesses which notes were played, when they started, when they stopped, and how hard they were struck. The best open-source model available right now achieves a 96.72% F1 score on the MAESTRO benchmark dataset, which sounds impressive until you realize that the remaining 3.28% of errors means roughly 1 in 30 notes is wrong, missing, or phantom — and in a dense chord passage, one wrong note can ruin an entire measure.

No tool on the market — free, paid, open-source, or cloud-based — reaches 100% accuracy. Every converted MIDI file needs manual cleanup. The question is not "can I convert MP3 to MIDI?" but "how much manual work am I signing up for?"

The model behind most "AI" converters

The majority of AI-driven MP3-to-MIDI tools released in the last three years — including browser-based services, standalone GUI applications, and VST plugins — trace back to one of two open-source projects: ByteDance's piano_transcription (released 2020, Apache 2.0 license) or Spotify's Basic Pitch. If a tool claims "AI-powered audio-to-MIDI conversion" and supports polyphonic input, there is a strong chance it wraps one of these two models.

The ByteDance model deserves specific numbers. On the MAESTRO evaluation dataset — a collection of virtuoso piano performances recorded on a Yamaha Disklavier — it achieved note F1 = 0.9677 and pedal F1 = 0.9186. The model file is literally named note_F1=0.9677_pedal_F1=0.9186.pth. These F1 scores surpassed Google's system, which scored 94.80% on the same dataset. The model recognizes 128 levels of velocity per note and detects sustain pedal onsets and releases. It broke through a previous 32-millisecond detection precision limit.

But MAESTRO is clean, professionally-recorded solo piano. Feed it a phone recording of a guitar, a muddy mix with vocals and drums, or even a piano recording with heavy room reverb, and that 96.72% number drops sharply. Nobody publishes the degraded numbers.

The dependency hell you only learn the hard way

Python version: 3.7, not 3.10

Multiple deployment tutorials independently agree: use Python 3.7 — specifically either 3.7.3 or 3.7.9. One tutorial author states "最好使用这个版本" (best to use this version) when specifying 3.7.3. Another chose 3.7.9 because the project uses f-strings, which require Python 3.6+. A third author tried Python 3.10 first and hit "各种报错" (various errors) before realizing version choice was critical.

The reason: the project's PyTorch dependency chain was built around the Python 3.7 ecosystem. Newer Python versions pull in incompatible library versions, especially for the librosa audio processing library.

The librosa version trap

If you install piano_transcription_inference without first pinning the librosa version, you will get this error:

No librosa.core attribute audio

This happens because librosa updated its API and removed or relocated the librosa.core.audio attribute. The fix is to install librosa first, pinned to a specific version:

pip install librosa==0.9.2
pip install piano_transcription_inference

Order matters. Install librosa 0.9.2 before the transcription package, not after. This bug was documented in GitHub issue #16 of the inference library repository.

PyTorch and CUDA: a 2 GB download with a trap door

The CUDA-enabled PyTorch installation command is:

pip install torch===1.7.0+cu110 torchvision===0.8.1+cu110 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

That is approximately 2 GB of download. If your network is unstable and the install fails midway, you need the offline package: torch-1.7.0+cu110-cp37-cp37m-win_amd64.whl, installed via:

pip install .\torch-1.7.0+cu110-cp37-cp37m-win_amd64.whl

The CUDA toolkit itself (version 11.1, approximately 3 GB) must be installed with a custom installation — only check the CUDA component items. Checking other components in the installer may cause the entire installation to fail. This is not documented in the official CUDA docs; it is the kind of thing you learn after a failed first attempt.

ffmpeg 4.3.1 and the path problem

The transcription pipeline requires ffmpeg version 4.3.1 (specifically the ffmpeg-4.3.1-2020-10-01-full_build package) to read media files. You must add ffmpeg's bin directory to your system PATH environment variable. If the path is wrong or ffmpeg is missing, you will get NoBackendError — the same error that appears when an entire dependency chain is missing from a packaged application (more on this below).

The NoBackendError: a real debugging chronicle

One user packaged the ByteDance transcription model into a GUI application with adjustable parameters and shared it on a community forum. The GUI exposed these parameters:

Another user downloaded the packaged GUI and ran it. It failed. They tried multiple audio formats. They tried the GPU version. They tried the CPU version. Every attempt produced this traceback:

[10:41:32] 参数: {'device': 'auto', 'sample_rate': 16000, 'batch_size': 8, 'segment_seconds': 10.0, 'onset_threshold': 0.5, 'offset_threshold': 0.5, 'frame_threshold': 0.5, 'velocity_scale': 1.0, 'enable_pedal': True}
[10:41:32] Checkpoint path: E:\setup\PianoTranscriptionGUI-CPU\PianoTranscriptionGUI\model\note_F1=0.9677_pedal_F1=0.9186.pth
[10:41:32] Using cpu for inference.
[10:41:34] Using CPU.
[10:41:34] 转录失败：
Traceback (most recent call last):
  File "app.py", line 465, in _run_transcription
  File "piano_transcription_inference\utilities.py", line 508, in load_audio
  File "audioread\__init__.py", line 132, in audio_open
audioread.exceptions.NoBackendError

The user noted that the older command-line pianotrans tool "一直用着没有什么问题" (had always worked fine with no problems), meaning the model and the code were not the issue — the packaging was.

The developer's diagnosis: "应该是缺少什么依赖打包进去了" (probably some dependency wasn't packaged in). They released a fixed version. The same user tried it — still failed, same NoBackendError, same traceback, just with a different path (fixed1 instead of the original).

The developer then said: "你可能得用另一台没有配置依赖库的电脑或者虚拟机来测试才行" — meaning the developer's own machine had the dependency already installed system-wide, so their local testing never caught the missing dependency in the packaged version. This is a classic "works on my machine" failure mode.

A second fixed version (fixed2) was released. This time the developer tested on a clean environment, confirmed no errors, and released it. The user confirmed: it worked. From initial release to working version, three repackaging rounds were needed — all because the audioread library's audio backend was not properly bundled.

Melodyne's locale bug: when your system language breaks the software

Melodyne 5.4 has a specific initialization bug that affects users whose Windows system locale is set to Chinese (Simplified, Singapore) or Chinese (Simplified, Hong Kong). These two locale variants have no corresponding lookup table in the Melodyne 5.4 code branch, so the software fails to initialize and displays the error:

detected in internal inconsistency and should be restarted to avoid data loss

The fix involves two approaches:

Approach 1 — Compatibility mode: Right-click the Melodyne executable → Properties → Compatibility → check "Run this program in compatibility mode" → try each Windows version in the dropdown. Multiple users report that Windows XP (Service Pack 3) mode resolves the issue.

Approach 2 — Locale fix: Press Win+R, type intl.cpl, press Enter. In the Format tab, select Chinese (Simplified, China) — even if it is already selected, reselect it and click Apply. Switch to the Administrative tab → Change system locale → select Chinese (Simplified, China) → check "Beta: Use Unicode UTF-8 for worldwide language support" → OK. Restart the computer (mandatory).

The locale fix is the more reliable solution because it addresses the root cause rather than masking it with compatibility shims.

Post-conversion reality: the work is not done

BPM mismatch

When the transcribed MIDI is imported into a DAW, the detected BPM often does not match the original. One user found that a song originally at 130 BPM was detected as 170 BPM after conversion. All note durations needed to be scaled by a factor of 170/130. Most DAWs do not have a "scale all note durations by ratio" feature.

The workaround: create a single sustained note lasting exactly 130 measures. Select all MIDI notes and stretch them until that reference note reaches 177 measures (130 × 170/130 ≈ 170, but the actual ratio needed was 177/130 due to timing rounding). This is a manual, imprecise process.

Pedal detection is visible but imperfect

The transcription model detects sustain pedal events. In the DAW piano roll, pedal releases appear as grey lines. You can see where the pedal was lifted, but the detection is not sample-accurate — pedal events may be slightly early or late, and you need to verify them by ear.

Note cleanup

Converted MIDI typically needs:

Sample rate trade-off

Higher sample rates capture more detail but slow down inference. The model was trained at 16000 Hz, and 16000 is the recommended setting. One user noted that 48000 Hz is "best not to change" but also admitted "我也没玩懂" (I don't really understand it either). The honest recommendation: use 16000 unless you have a specific reason to do otherwise.

Mac vs Windows: different tools, same pain

Windows: ByteDance piano_transcription ecosystem

The ByteDance model ecosystem on Windows involves:

Mac: MuScriptor-based tools

A separate Mac-only application was built on the open-source MuScriptor project (different from ByteDance's piano_transcription). It ships with:

On an M1 Mac Studio, converting a 3-minute audio file with the largest model takes approximately 3 minutes.

Known issues with the Mac tool:

Where sources contradict each other

Contradiction 1 — Python version: One source specifies Python 3.7.3 as "the best version to use." Another uses 3.7.9. A third attempted 3.10 and hit multiple errors. All agree 3.7 is the correct major version, but the minor version recommendation varies — and nobody explains why one minor version would matter over another.

Contradiction 2 — GPU conversion speed: One source claims GPU inference takes "about 10 seconds to 1+ minute per song" on an NVIDIA GPU. Another reports 3 minutes for a 3-minute song on an M1 Mac Studio with the largest model. The first figure is likely for shorter or less complex pieces, while the second is for a full-length song with the largest available model. Neither source specifies the GPU model or CUDA core count, making direct comparison impossible.

Contradiction 3 — The same model, different results: The note_F1=0.9677_pedal_F1=0.9186.pth model file works perfectly in a command-line environment but throws NoBackendError in a packaged GUI — same model, same parameters, same machine. The failure was not caused by the model or the code but by a missing audio backend dependency that was present in the developer's environment but absent in the user's. This means that "the model works" is not the same as "the tool works."

Contradiction 4 — .wma support: The original open-source project supports .wma files. A repackaged GUI version does not. The repackaging process — intended to make the tool easier to use — inadvertently removed codec support. Users who rely on .wma files need to go back to the command-line version.

Contradiction 5 — Sample rate 48000: One user recommends leaving 48000 Hz alone but admits not understanding why. The model was trained on 16000 Hz data. Using 48000 Hz means the input must be downsampled internally, which could introduce artifacts or simply waste processing time. Nobody has tested whether 48000 produces better or worse results than 16000.

What converts well, what doesn't

Converts well:

Converts poorly:

The cloud alternative (and why it might not be enough)

Browser-based tools exist that require no installation:

These tools are convenient but inherit the same accuracy limitations as the open-source models they wrap. They also introduce privacy concerns — your audio is uploaded to a server — and may have file size or duration limits that make them impractical for longer recordings.

The honest verdict

MP3-to-MIDI in 2026 is at the stage where the technology works well enough to be useful but not well enough to be trusted blindly. The best model achieves 96.72% F1 on idealized test data. In real-world conditions with non-piano input, room reverb, or mixed ensembles, expect 70-85% accuracy — which means budgeting 15-30 minutes of manual cleanup per converted song.

The deployment story is equally honest: if you use a cloud tool, you trade control and privacy for convenience. If you self-host, you sign up for Python 3.7, version-pinned dependencies, CUDA installation traps, and the ever-present risk that a repackaged version is missing a dependency you will only discover at the moment you need it.

The people who built GUI wrappers and deployment tutorials around this model did so because they hit these walls themselves. The NoBackendError traceback, the librosa version conflict, the locale bug in Melodyne 5.4, the model degradation cascade — these are not hypothetical scenarios. They are the records of actual failures encountered by actual users who then wrote down what they learned. That is the only kind of documentation that matters.