Technical Review: Mel-Frequency Cepstral Coefficients

Mel-Frequency Cepstral Coefficients (MFCCs): Technical Review

Review scope:The MFCC literature and current official documentation/source for Kaldi, librosa 0.11.0, and TorchAudio 2.11, reviewed on 8 July 2026.

Executive assessment

MFCCs are not one universally standardized formula. They are a family of short-time acoustic front ends built around a common sequence:

waveform

-> framing and windowing

-> magnitude or power spectrum

-> Mel-spaced spectral pooling

-> logarithmic or decibel compression

-> discrete cosine transform

-> selection of cepstral coefficients

Pre-emphasis, dithering, DC removal, frame-energy substitution, liftering, delta features, and cepstral normalization are common additions, but none is required by the bare concept of an MFCC. Modern toolkits expose materially different defaults, so two feature matrices can both be called “MFCCs” without being numerically interchangeable.[^2][^3][^6][^12]

The representation is best understood as a purposeful information bottleneck. It emphasizes broad, short-time spectral shape while reducing phase information, fine frequency detail, and the dimensionality of the spectral representation. This is often useful for speech and for compact acoustic baselines. It is not a neutral description of sound, a direct measurement of vocal-tract state, or a universally appropriate representation for every audio task.

Three corrections are especially important:

1.The DCT does not by itself reduce dimensionality.A full DCT is a change of basis. Dimensionality is reduced when only a subset of its coefficients is retained.[^8]

2.A complete MFCC front end is not necessarily deterministic.Kaldi enables random waveform dithering by default; deterministic extraction requires disabling or controlling that stage.[^2][^5]

3.“Log Mel spectrum” is not one numerical convention.Kaldi uses logarithms of Mel-band energies, while current librosa and default TorchAudio MFCC paths use decibel-scaled Mel power with floors and clipping conventions that can change results.[^9][^12]

1. What an MFCC front end actually computes

For frame \(m\), let the windowed waveform be

\[

x_m[n]= x[n+mR]w[n],

\]

where \(R\) is the hop size and \(w[n]\) is the analysis window. An FFT produces complex coefficients \(X_m[k]\). Most MFCC pipelines discard phase and continue with a magnitude- or power-like spectrum

\[

P_m[k]= |X_m[k]|^p,

\]

commonly with \(p=2\).

A Mel filter bank pools this spectrum:

\[

E_m[r]= \sum_k H_r[k]P_m[k],

\]

where \(H_r[k]\) is the weight of Mel band \(r\). A logarithmic or decibel transform produces \(L_m[r]\), and a DCT rotates the Mel-band vector into cepstral coordinates:

\[

c_m[q] = \sum_{r=0}^{M-1}L_m[r]

\cos\left(\frac{\pi q}{M}\left(r+\frac{1}{2}\right)\right).

\]

If all \(M\) DCT coefficients are retained with a compatible normalization, this stage is invertible as a linear transform. The usual compression occurs because the system keeps only the first \(Q<M\)coefficients.Currentlibrosasourcemakesthissequenceexplicit:itcomputesaDCTandthenslicestheresultton_mfccrows.[^8]

Thisdistinctionmatters.Itisinaccuratetosaythat“theDCTreducesa40-bandMelspectrumto13values.”TheDCTchangescoordinates; DCTtruncationreducesthenumberofvalues.

##2.Framing,windowing,andfront-endpreparation

A20–30msframewitharoughly10msshiftremainsacommonspeechconfiguration,butitisaconventionratherthanpartofthemathematicaldefinition.At16kHz,25mscorrespondsto400samplesanda10mshopto160samples.

Kaldi’scurrentdefaultframeconfigurationisunusuallyspecific:16kHzsampling,a25msframe,a10msshift,DC-offsetremoval,pre-emphasiscoefficient0.97,randomditherwithamplitude1.0,aPoveywindow,power-of-twoFFTpadding,andsnip_edges=true.[^2]ThePoveywindowisHann-likeandraisedtoapowerof0.85;HammingissupportedbutisnotKaldi’sdefault.DescribingKaldi’sordinarypipelineasusingaHammingwindowisthereforeincorrect.[^2]

Otherlibrariesuseverydifferentgeneral-purposedefaults.librosa0.11.0defaultsto22,050Hz, n_fft=2048, hop_length=512,aHannwindow,centeredframes,andzeropaddingattheedges.[^7]Thosesettingscorrespondtoapproximately92.9mswindowsand23.2mshopsatthedefaultsamplingrate,whicharenotaconventional25ms/10msspeechsetup.TorchAudio’sunderlyingMelSpectrogramdefaultsto16kHz,a400-sampleFFT/window,a200-samplehop,aHannwindow,centeredframing,andreflectededgepadding.[^13]

Consequently,areproducibleMFCCdescriptionmustincludemorethanframelengthandhop.Itshouldalsostate:

-whetherframesarecenteredorbeginatfixedsampleoffsets;

-howthesignalispaddedatthebeginningandend;

-whetherincompleteedgeframesarediscarded;

-thewindowtypeanditsprecisedefinition;

-whethertheframeiszero-paddedbeforetheFFT;

-whetherDCremoval,pre-emphasis,orditherisapplied.

###Ditheringanddeterminism

ThemathematicalMFCCtransformisdeterministiconcetheinputandalloperationsarefixed.Acompletetoolkitpipelinemaynotbe.KaldiaddsGaussianditherbydefault,anditsFAQexplicitlynotesthatrepeatedfeatureextractioncanproducesmallnumericaldifferencesforthisreason.Kaldirecommends--dither=0togetherwithanenergyfloorwhendeterministicoutputisrequired.[^2][^5]

Themoreaccurateclaimistherefore:

>MFCC extraction can be deterministic, but exact reproducibility depends on whether the chosen front end contains stochastic preprocessing.

3. Spectrum and Mel filter-bank conventions

MFCC implementations usually operate on magnitude or power after discarding Fourier phase. Power is the common default in Kaldi, librosa, and TorchAudio’s standard paths.[^7][^13][^14]Phase loss is real and irreversible: two signals with similar short-time magnitude spectra but different phase can map to similar MFCCs.

A larger zero-padded FFT gives a denser sampling of the spectrum, but it does not provide the physical frequency resolution of observing a longer time interval. The analysis-window duration sets the main time-frequency tradeoff; zero padding mainly interpolates the sampled spectrum.

“The Mel scale” is not one scale

Kaldi uses

\[

m = 1127\ln(1+f/700),

\]

which is algebraically equivalent to the common HTK-style \(2595\log_{10}(1+f/700)\) expression.[^4]librosa defaults to the Slaney/Auditory Toolbox convention and supports the HTK formula throughhtk=True; TorchAudio’sMelSpectrogramdefaults tomel_scale="htk".[^10][^13]

The filters also differ in normalization. librosa defaults to Slaney-style area normalization, while TorchAudio defaults to no filter normalization unlessnorm="slaney"is requested.[^10][^13]Kaldi constructs triangular bands in Mel coordinates and does not expose librosa’s area-normalization convention as the ordinary default.[^4]

Default band counts and frequency limits differ sharply

Kaldi’s MFCC configuration defaults to 23 Mel bands and 13 cepstral outputs. Its default lower cutoff is 20 Hz, andhigh_freq=0means the upper cutoff is Nyquist—not 7.8 kHz.[^3][^4]At 16 kHz, Nyquist is 8 kHz.

librosa’s general-purpose Mel spectrogram defaults to 128 bands, 0 Hz to Nyquist, Slaney spacing, and area normalization.[^7][^10]TorchAudio’s default Mel spectrogram also uses 128 bands from 0 Hz to Nyquist, but uses HTK spacing and no band normalization.[^13]

These are not minor implementation details. They change the frequency grid, the relative scale of band energies, and every downstream cepstral coefficient.

4. Logarithms, decibels, floors, and clipping

A textbook description often applies a natural logarithm to each Mel-band energy:

\[

L_m[r]= \ln(E_m[r]+ \epsilon).

\]

Kaldi follows the log-energy form in its MFCC computation.[^14]Current librosa behaves differently when a waveform is passed directly: it computes a Mel power spectrogram, appliespower_to_db, then applies the DCT.[^8]power_to_dbuses \(10\log_{10}\), a minimum floor, and by default clips values to 80 dB below the peak.[^9]

TorchAudio also defaults to a dB-scaled Mel spectrogram rather than the simplest textbook log. Its documentation warns that the output depends on the maximum value in the input spectrogram, so processing a full recording can differ from processing the same audio in separate snippets.[^12]

librosa deserves a related qualification. Its default reference is fixed at 1.0, but itstop_db=80floor is defined relative to the peak of the supplied spectrogram. Low-energy values that hit that floor can therefore depend on whether the feature extractor sees a full clip, a channel group, or a shorter segment.[^9]

Without floors, clipping, or reference changes, natural-log power and decibel power mainly differ by a constant scale factor. Real implementations add floors, peak-relative clipping, energy replacement, and normalization, so they are not generally reducible to one harmless constant.

A complete methods section should state:

-natural logarithm or decibels;

-power or magnitude input;

-log base or dB multiplier;

-numerical floor;

-reference value;

-dynamic-range clipping such astop_db;

-whether scaling is performed per clip, per channel, or over a larger batch.

5. The DCT, coefficient selection, C0, energy, and liftering

The DCT is useful because smooth log-Mel spectra are often represented compactly by low-order cosine patterns. Low orders describe broad variation across the Mel axis, while higher orders describe progressively finer oscillation. This is a useful architectural interpretation, but individual coefficients should not be given rigid universal meanings such as “C1 is a formant coefficient” or “C2 is a speaker coefficient.” Every coefficient combines information from the full Mel-band vector.

The DCT can reduce correlation in practice, but it does not guarantee statistical independence or optimal decorrelation for every corpus. It is a fixed transform, not a dataset-fitted PCA.

C0 is not identical to frame energy

The zeroth DCT coefficient is proportional—subject to normalization—to the sum or average of log Mel-band values. It is related to overall spectral level, but it is not the same quantity as the time-domain energy of the frame.

Kaldi makes the distinction explicit. It includes C0 in its DCT matrix, but by default setsuse_energy=trueand replaces C0 with log frame energy. It also defaults toraw_energy=true, so that energy is calculated before pre-emphasis and windowing. Kaldi uses 13 outputs and a default cepstral lifter of 22.[^3][^14]

Thus, two 13-dimensional vectors can differ semantically:

C0, C1, ..., C12

versus

log frame energy, C1, ..., C12

The coefficient ordering can also change under HTK-compatibility settings.[^3]

Liftering changes feature weighting

Liftering multiplies cepstral coefficients by an order-dependent weighting function. It does not restore discarded information and should not be described as universally “equalizing” coefficient ranges. librosa notes that sufficiently large lifter values emphasize higher-order coefficients; Kaldi describes its lifter as a scaling convention.[^3][^6]

6. Source-filter motivation: useful, but only approximate

A classical speech model writes the waveform as excitation convolved with a vocal-tract filter:

\[

x(t)=e(t)*h(t),

\]

so in the frequency domain

\[

X(f)=E(f)H(f).

\]

Taking the logarithm of a high-resolution magnitude spectrum converts multiplication into addition. This is the conceptual background of cepstral analysis.

MFCCs only approximate that reasoning. They first integrate power over broad, overlapping Mel filters and then take the logarithm:

\[

\log\left(\sum_k H_r[k]|E[k]H[k]|^2\right).

\]

The logarithm of this sum does not cleanly separate into an excitation term plus a vocal-tract term. Mel smoothing and low-order cepstral retention often emphasize broad spectral envelope, but MFCCs are not a literal physical decomposition of source and filter.

This qualification resolves several common overstatements:

-MFCCsreducesensitivity to fine harmonic structure; they do not remove pitch mathematically.

-MFCCs contain substantial speaker information because vocal-tract length, spectral tilt, and formants affect the envelope.

-Log compression reduces amplitude range; it does not guarantee loudness invariance.

-Low-order coefficients describe broad log-Mel shape; they do not directly report formant frequencies.

7. Temporal information, deltas, and normalization

A ten-second recording does not normally produce one MFCC vector. It produces a matrix with coefficient and time axes. With a 10 ms hop, ten seconds yields approximately one thousand frame positions.

Each static vector is local, but thesequenceretains the evolution of those local observations. It is therefore too strong to say that framing itself “discards long-range context.” The base front end does not encode long-range context inside one frame vector, but a downstream HMM, recurrent network, Transformer, temporal convolution, or statistical pooling stage can use the full sequence.

Delta and delta-delta features are separate temporal augmentations. They approximate local first- and second-order changes across neighboring MFCC frames. A traditional regression delta is

\[

\Delta c_t =

\frac{\sum_{n=1}^{N}n(c_{t+n}-c_{t-n})}

{2\sum_{n=1}^{N}n^2},

\]

but this is a convention rather than a universal definition. Kaldi computes deltas in a separate feature-processing program, while librosa's currentfeature.deltauses a Savitzky–Golay derivative filter with configurable width, order, and boundary behavior.[^15][^16]

Cepstral mean normalization is commonly written as

\[

c'_t = c_t - \frac{1}{T}\sum_{\tau=1}^{T}c_\tau.

\]

Cepstral mean-and-variance normalization may additionally divide each coefficient by an estimated standard deviation. The statistics can be computed per utterance, speaker, recording, or window, and that scope changes the resulting features. Kaldi treats mean subtraction and CMVN as operations outside base MFCC extraction.[^17]These operations can reduce persistent channel coloration, but they do not remove arbitrary additive noise or reverberation. An accurate methods section should distinguish:

static MFCCs

from

MFCCs + deltas + delta-deltas + CMVN

because these are different feature pipelines.

8. What MFCCs preserve—and what they do not

MFCCs are effective at representing broad, short-time spectral shape. Depending on configuration, they preserve information related to spectral slope, broad resonance structure, timbre, phonetic content, speaker characteristics, and overall energy.

They deliberately or indirectly reduce:

-waveform phase;

-fine linear-frequency resolution through Mel pooling;

-fine variation in the log-Mel vector through coefficient truncation;

-precise harmonic and pitch structure;

-information outside the selected frequency range;

-context beyond the local frame unless a downstream temporal model is used.

They are also sensitive to additive noise and reverberation because both alter the short-time spectrum. The Mel scale and logarithm do not identify which energy came from speech and which came from traffic, a fan, another speaker, or a room response.

For environmental sound or music, the discarded information can be central. Tasks driven by precise pitch, narrow tonal peaks, phase relationships, impulsive transients, or high-frequency detail may be better served by log-Mel spectra, constant-Q features, pitch representations, waveform models, or learned front ends. MFCCs can still be a useful baseline, but their suitability must be tested rather than assumed.

9. Reconstruction and invertibility

Compact MFCCs are not uniquely invertible. Phase has already been discarded, Mel filters pool several FFT bins together, and cepstral truncation removes dimensions from the log-Mel representation.

Approximate reconstruction is possible. librosa’s inverse utilities apply an inverse DCT to approximate the Mel power spectrogram, map that representation back toward a linear-frequency magnitude estimate, and use iterative phase reconstruction to synthesize audio.[^11]The result can demonstrate what the representation retained, but it is not recovery of the original waveform.

Even retaining all DCT coefficients would only remove the loss caused by cepstral truncation. It would not restore original phase or spectral detail destroyed by Mel pooling.

10. Current toolkit defaults are not interchangeable

The following table summarizes ordinary documented defaults, not recommended settings for every task.

Sources: Kaldi frame, Mel, and MFCC options; librosa MFCC/Mel documentation and source; TorchAudio MFCC/MelSpectrogram documentation.[^2][^3][^4][^6][^7][^8][^10][^12][^13]

This table explains why “we extracted 13 MFCCs” is not reproducible methodology. Toolkits can be configured to resemble one another more closely, but matching onlyn_mfccis insufficient.

Multi-channel audio

MFCCs are most straightforwardly defined per channel. librosa warns that its multi-channel MFCC result can depend on peak loudness across the supplied channels and may differ from calculating each channel independently.[^6]A stereo or microphone-array workflow should therefore state whether channels were analyzed separately, downmixed, beamformed, or jointly scaled.

11. What the historical evidence actually established

Davis and Mermelstein’s 1980 comparison is rightly influential, but it should not be summarized as universal proof that MFCCs are the best speech representation. Their particular speaker-dependent word-recognition experiment used ten MFCCs, a 25.6 ms analysis window, and a 6.4 ms frame separation. In the reported open tests, MFCCs achieved 96.5% and 95.0% for the two speakers and outperformed the other representations tested in that system.[^1]

The paper itself cautioned that the results were directly applicable to the specific recognition system and that generalization to differently organized systems might not be warranted.[^1]The experiment involved two speakers, a constrained vocabulary, dynamic time warping, and recording conditions that are very different from modern large-vocabulary, multilingual, noisy, or neural speech systems.

The appropriate historical conclusion is:

>MFCCs were highly effective in an influential early comparison and became a durable engineering choice—not that one 1980 experiment established universal optimality.

12. Reproducibility and evaluation

A defensible MFCC methods section should record at least:

toolkit:librosa / Kaldi / TorchAudio / other

version_or_commit:

input:

sample_rate:

resampling_method:

channel_handling:

framing:

frame_length_samples_or_ms:

hop_samples_or_ms:

center_frames:

padding_or_snip_edges:

preprocessing:

dither:

random_seed_or_rng_policy:

remove_dc:

pre_emphasis:

window:

fft_size:

spectrum:

magnitude_exponent:

mel:

formula:HTK / Slaney / other

number_of_bands:

fmin:

fmax:

filter_normalization:

compression:

natural_log_or_db:

reference:

numerical_floor:

top_db:

cepstrum:

dct_type:

dct_normalization:

n_mfcc:

coefficient_order:

c0_or_frame_energy:

lifter:

postprocessing:

deltas:

delta_deltas:

CMN_or_CMVN:

utterance_pooling:

numerics:

dtype:

device:

Evaluation should test the complete front end on the downstream task. Useful ablations include frame duration, hop, Mel convention, band count, retained coefficient count, pre-emphasis, C0 versus energy, liftering, normalization, and delta features. Noise, reverberation, channel response, segment length, and multi-channel handling should be stress-tested separately.

Overall assessment

MFCCs remain one of the clearest examples of successful hand-designed feature engineering. Their value comes from an explicit sequence of assumptions:

-analyze sound locally in time;

-represent each frame spectrally;

-discard phase;

-pool frequency with a Mel-spaced bank;

-compress energy nonlinearly;

-express broad Mel-spectrum patterns in a cosine basis;

-retain only the coefficient orders judged useful.

This architecture is compact, computationally inexpensive, easy to inspect, and often effective when broad spectral-envelope dynamics matter. Its limitations follow directly from the same choices. Fine pitch, phase, narrow spectral structure, long-range context, and exact waveform identity are not preserved by default.

The right technical question is not whether MFCCs are “outdated.” It is whether the information retained by a precisely specified MFCC front end matches the information required by the task. That question cannot be answered from the labelMFCCalone.

References

[^1]: S. B. Davis and P. Mermelstein, “Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Sentences,”IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(4), 1980. https://doi.org/10.1109/TASSP.1980.1163420 ; accessible PDF: https://courses.grainger.illinois.edu/ece417/fa2017/davis80.pdf

[^2]: Kaldi,FrameExtractionOptionsand window source. https://kaldi-asr.org/doc/feature-window_8h_source.html

[^3]: Kaldi,MfccOptions. https://kaldi-asr.org/doc/structkaldi_1_1MfccOptions.html

[^4]: Kaldi, Mel-bank options and Mel-scale source. https://kaldi-asr.org/doc/mel-computations_8h_source.html ; https://kaldi-asr.org/doc/mel-computations_8cc_source.html

[^5]: Kaldi FAQ, “Indeterminacy in feature extraction.” https://kaldi-asr.org/doc/faq.html

[^6]: librosa 0.11.0,librosa.feature.mfcc. https://librosa.org/doc/0.11.0/generated/librosa.feature.mfcc.html

[^7]: librosa 0.11.0,librosa.feature.melspectrogram. https://librosa.org/doc/0.11.0/generated/librosa.feature.melspectrogram.html

[^8]: librosa 0.11.0 MFCC implementation source. https://librosa.org/doc/0.11.0/_modules/librosa/feature/spectral.html

[^9]: librosa 0.11.0,librosa.power_to_db. https://librosa.org/doc/0.11.0/generated/librosa.power_to_db.html

[^10]: librosa 0.11.0,librosa.filters.mel. https://librosa.org/doc/0.11.0/generated/librosa.filters.mel.html

[^11]: librosa 0.11.0, inverse MFCC utilities. https://librosa.org/doc/0.11.0/generated/librosa.feature.inverse.mfcc_to_audio.html ; https://librosa.org/doc/0.11.0/generated/librosa.feature.inverse.mfcc_to_mel.html

[^12]: TorchAudio 2.11,torchaudio.transforms.MFCC. https://docs.pytorch.org/audio/stable/generated/torchaudio.transforms.MFCC.html

[^13]: TorchAudio 2.11,torchaudio.transforms.MelSpectrogram. https://docs.pytorch.org/audio/stable/generated/torchaudio.transforms.MelSpectrogram.html

[^14]: Kaldi MFCC implementation source. https://kaldi-asr.org/doc/feature-mfcc_8cc_source.html ; https://kaldi-asr.org/doc/classkaldi_1_1MfccComputer.html

[^15]: Kaldi feature-extraction documentation andadd-deltas. https://kaldi-asr.org/doc/feat.html ; https://kaldi-asr.org/doc/add-deltas_8cc.html

[^16]: librosa 0.11.0,librosa.feature.delta. https://librosa.org/doc/0.11.0/generated/librosa.feature.delta.html

[^17]: Kaldi, feature transforms and CMVN. https://kaldi-asr.org/doc/transform.html