ITSimplera Internship — Six Weeks, Six Deliverables
From first EDA to a fine-tuned transformer
Results — hover any figure for the baseline and the caveat
Macro-F1, 5 sections
vs TF-IDF + logistic regression
Macro-F1 weights all five sections equally regardless of class size, which is why it's the ranking metric rather than accuracy.
LDA topic purity
vs NMF cross-check
Two different algorithms on different features recovering the same structure is the strongest available evidence the topics are real and not one model's inductive bias. AMI 0.740, ARI 0.750.
Entity mentions extracted
21,669 unique across 18 types, 38.7 per article. Normalised per-article rather than raw counts — raw counts would just re-measure how many articles each section has.
kWh RMSE, energy model
vs 11 PCA components at 95% variance
R² of 0.9987 flatters the model because R² compares against the mean. RMSE tripling under PCA is the honest signal.
Training cost of the winner
vs linear baseline
22.9 minutes versus 0.8 seconds, for +2.0 macro-F1. Figure 8 in the week 6 README plots this trade-off deliberately so it can't be buried.
The problem
A six-week internship brief that could have been six notebooks. The risk with weekly ML assignments is that each one ends at a number in a cell output, with no held-out discipline, no baseline, and nothing anyone else can run.
What I built
I treated each week as a shippable repo: README with results, notebooks that run top to bottom, figures generated by script, and an app someone non-technical can open. Every modelling week declared a split before fitting, kept a baseline in the table even when it lost, and reported what the winning model cost as well as what it scored.
The table
| Model | Family | Accuracy | Macro-F1 | Train cost |
|---|---|---|---|---|
| DistilBERT fine-tuned | Transfer (fine-tuned) | 98.33% | 0.9830 | 22.9 min |
| DistilBERT features + LR | Transfer (frozen) | 96.66% | 0.9653 | 6.4 min |
| TF-IDF + logistic regression | Classical | 96.42% | 0.9629 | 0.8 s |
| TF-IDF + XGBoost | Classical | 93.79% | 0.9367 | 117 s |
Week 6, BBC News. One stratified 70/10/20 split, 419-article test set untouched until final scoring.
What the results actually say
The transformer wins, but the margin is small and the cost is not.
+2.0 macro-F1 for 1,700× the training time and a 268 MB artefact instead of 12 MB. If the deployment constraint were latency or footprint rather than accuracy, the honest recommendation would be the linear model.
Frozen embeddings barely beat TF-IDF, by +0.24 F1.
This is the most informative number in the table: the pre-trained representations alone are worth almost nothing here over a good bag of words. Running both strategies is what makes the win attributable to fine-tuning rather than to 'BERT' in general.
XGBoost lost to logistic regression.
On 40,530 sparse TF-IDF features with 1,464 training rows, trees must carve axis-aligned splits through a space where the linear boundary is already close to correct. High-dimensional sparse text is the one regime where the simpler model is genuinely better.
Perplexity was computed, plotted, and deliberately not used to select k.
It is known to anti-correlate with human interpretability (Chang et al., 2009). Plotting a metric and then declining to select on it is the part of this work I'd most want to be asked about.
What this doesn’t prove
- BBC News is a clean, balanced, well-separated benchmark. 98% accuracy on it says much less than the same number would on messy production text.
- Relevance and topic labels come from the corpus's own editorial sections, so 'purity' measures agreement with one newsroom's taxonomy, not ground truth.
- The energy models are trained on a single plant's 2018 data; nothing here demonstrates transfer to another site.
- No monitoring or retraining loop was deployed for these — they are batch results with apps in front of them, not running services.
This section is here on purpose. A result without its limits isn’t a result, it’s a claim.