Ask Jev once
A typed Noul question gives each abstract a relevance score. Keep these judgments in a cache.
A Jev + active learning experiment
Read less.
Find the evidence.
One ranking learns from you. Another brings Jev’s prior judgment. Combine them to choose the next abstract worth reading.
No API key. No sign-up. Every curve comes from saved benchmark runs.
13validation reviews
11 / 13reviews improved
+0.075mean WSS@95 gain
78matched control traces
01 / Open the evidence
Every gain. Both losses.
Replay the same review, starting from the same labeled examples. Move the reading budget to compare what each method finds.
Relevant studies found
ELAS u4 Jev + ELAS u4 – – 95% target
Drag or use arrow keys. This is a replay of saved decisions, not live inference or a stopping recommendation.
The initial review is the median by improvement across the 13 validation reviews. No examples are hidden.
All 13 reviews Mean WSS@95 across three seeds · click a review to replay
| Review | ELAS u4 | Jev + ELAS u4 | Difference |
|---|
02 / The reusable idea
A prior, then a partnership.
Jev scores are cached once. The local model learns from each human decision. Fuse their rankings over the unread pool and repeat.
Learn from the reader
ELAS u4 retrains after each label. Its classifier, text features and initial examples stay unchanged.
Combine the ranks
Equal-weight reciprocal rank fusion selects the next abstract. No extra API call is needed per decision.
Zero runtime dependencies
Bring your own ranker.
The small Python adapter works with matching record IDs. ASReview is needed only to reproduce the research simulations.
Integration guide ↗from jev_shortlist import rank_records
# Same unread record IDs in both mappings.
queue = rank_records(local_scores, cached_jev_scores)
next_record = queue[0]
# Label, retrain, remove the read record, repeat.03 / What this does and doesn’t show
Useful evidence.
With visible limits.
Full protocol and statistics ↗A specific comparison. ASReview 3.0.8, ELAS u4. Thirteen development reviews selected the method; thirteen others supplied this internal-validation result. All 26 had appeared in earlier project experiments.
A retrospective benchmark. The subsets are capped at 2,000 records, with all included records retained. Known labels let us locate 95% recall afterward. This does not tell a reviewer when it is safe to stop in practice.
Uncertainty stays in the picture. Mean WSS@95 gain +0.0748; review-level 95% bootstrap interval +0.0365 to +0.1183. Two reviews got worse. Independent reviews and full-size datasets remain untested.
Costs are separate. This fusion experiment used saved Jev judgments and made no new API calls. Acquiring those judgments originally had a cost.