Meet NeuroVFM: A New Baseline Model for Vol-JEPA-Trained Neuroimaging in Uncurated Clinical MRI and CT Volumes

admin 4 hours ago

0 0 3 minutes read

Meet NeuroVFM: A New Baseline Model for Vol-JEPA-Trained Neuroimaging in Uncurated Clinical MRI and CT Volumes

Frontier models learn a lot from public internet data. However, clinical neuroimaging is rarely performed there, because MRI and CT scans contain identifiable facial features. Therefore, standard models do not work well for brain imaging tasks. A team of researchers from the University of Michigan is tackling this gap NeuroVFMpublished in Nature Medicine.

What is NeuroVFM?

At its core, NeuroVFM is a standard visual basis model for neuroimaging. Specifically, 5.24 million MRI and CT clinics were trained. This comes from 566,915 subjects in the UM-NeuroImages dataset. That data spans more than two decades of standard care at Michigan Medicine.

The research team calls their approach ‘health system learning.’ In short, the model learns from unprocessed data generated during normal clinical practice. Therefore, it avoids the bottleneck of paired radiology reports. It also avoids the disease-specific processing used in smaller classifiers.

Notably, the basic model is called Vol-JEPA. It extends the previous I-JEPA and V-JEPA methods to quantitative medical imaging. This reflects a broader trend: JEPA-style learning extends to clinical reasoning.

How Vol-JEPA works?

Vol-JEPA is a self-regulating, theoretical algorithm. Rather than reconstructing pixels, it predicts representations from a learned hidden space. As a result, it requires no labels, no report text, and no voxel decoder.

First, each 3D volume is tokenized into non-overlapping 4×16×16-voxel patches. Next, the volume is divided into a small visible core and a large hidden target. The learner encoder then processes context patches.

Meanwhile, the predictor combines contextual and spatial encoding latencies. It predicts the hidden objects of the region. The teacher encoder creates a hidden target of ground truth. This teacher is the explanatory moving average (EMA) of the student. Training reduces the loss of L1 fluency between the predicted and the teacher’s latent, with gradients established by the teacher.

Essentially, masking is focused on the front, using a pre-made head mask. Context rates are 25% for MRI and 20% for CT, and 20% for patch dropout. This design encourages the coder to model the shared neuroanatomy instead of background shortcuts.

0;i–){var j=Math.floor(Math.random()*(i+1));var t=a[i]a[i]=a[j]a[j]=t;}return a;} function buildSplit(){ var pool=shuffle(fg.slice()); var nCtx=Math.max(3,Math.round(pool.length*ctxRatio[modality])); ctxSet=pool.slice(0,nCtx); // 20% drop patches for others -> still target; here all the rest of the foreground are targets tgtSet=pool.slice(nCtx); } function clearClasses(){cells.forEach(function(c){c.className=”cell”;});} function paintFg(){cells.forEach(function(c){if(c.dataset.fg===’0′)c.className=”cell”;});} var boxes={s:document.getElementById(‘bStudent’),p:document.getElementById(‘bPred’),t:document.getElementById(‘bTeacher’),l:document.getElementById(‘bLoss’)}; function boxesOff(){ Object.keys(boxes).forEach(function(k){ boxes[k].classList.remove(‘on’);});} var note=document.getElementById(‘note’); var pills=[].slice.call(document.querySelectorAll(‘.pill’)); function setStep(s){pills.forEach(function(p){p.classList.toggle(‘active’, +p.dataset.s===s);});} var lossval=document.getElementById(‘lossval’), lossfill=document.getElementById(‘loss) var loss=1.00; function updateLoss(){ lossval.textContent=loss.toFixed(2); lossfill.style.width=Math.max(6,loss*100)+’%’;} var stage=0; function render(){ clearClasses(); boxesClosed(); switch(stage){ case 0: // tokenize setStep(0); cells.forEach(function(c){ if(c.dataset.fg===’1′){ c.classList.add(‘pulse’); setTimeout(function(cc){return function(){cc.classList.remove(‘pulse’);};}(c), 250);}}); note.innerHTML=’Tokenize. Each 3D volume is divided into non-overlapping 4x16x16-voxel patches. Only the previous episodes are saved (in the head).’; a break; case 1: // context/target split setStep(1); buildSplit(); ctxSet.forEach(function(i){cells[i].classList.add(‘ctx’); }); tgtSet.forEach(function(i){cells[i].classList.add(‘tgt’); }); note.innerHTML=’Context / Target. Little visible context (blue) is taken. The big one is covered target (resin) is predicted.’; a break; case 2: // reader encodes the content setStep(2); ctxSet.forEach(function(i){cells[i].classList.add(‘ctx’,’pulse’); }); tgtSet.forEach(function(i){cells[i].classList.add(‘tgt’); }); boxes.s.classList.add(‘on’); note.innerHTML=’Student codes. The reader encoder Eθ converts visible context clips into context summaries.’; a break; case 3: // predict hidden target setStep(3); ctxSet.forEach(function(i){cells[i].classList.add(‘ctx’); }); tgtSet.forEach(function(i){cells[i].classList.add(‘tgt’,’pulse’); }); boxes.s.classList.add(‘on’); boxes.p.classList.add(‘on’); note.innerHTML=’Guess what is hidden. The Pφ predictor uses context hidden objects and target states to predict state hidden objects.’; a break; case 4: // teacher generates target setStep(4); cells.forEach(function(c){if(c.dataset.fg===’1′)c.classList.add(‘ctx’);}); tgtSet.forEach(function(i){cells[i].classList.remove(‘ctx’);cells[i].classList.add(‘tgt’,’filled’); }); boxes.t.classList.add(‘on’); note.innerHTML=’Teacher (EMA). The teacher’s encoder, the student’s moving average, produces a hidden ground truth target. Gradients are set here.’; a break; case 5: // set lossStep(5); ctxSet.forEach(function(i){cells[i].classList.add(‘ctx’); }); tgtSet.forEach(function(i){cells[i].classList.add(‘tgt’,’filled’); }); boxes.l.classList.add(‘on’); loss=Math.max(0.05, loss*0.82); updateLoss(); note.innerHTML=’Smooth L1 loss. Training reduces the distance between the predicted latencies and the teacher. Losses decrease in steps.’; a break; } postLength(); } function next(){ stage=(stage+1)%6; assign (); } var timer=null, playing=false; var playBtn=document.getElementById(‘play’); play play(){ play=true; playBtn.textContent=”❚❚ Pause”; timer=setInterval(next,1600); } function pause(){ play=false; playBtn.textContent=”▶ Play”; clearInterval(time); } playBtn.onclick=function(){ playing?pause():play(); }; document.getElementById(‘step’).onclick=function(){ if(playing)pause(); Next(); }; document.getElementById(‘reset’).onclick=function(){ if(playing)pause(); category=0; loss=1.00; updateLoss(); assign (); }; var metaEl=document.getElementById(‘meta’); document.getElementById(‘mod’).addEventListener(‘click’,function(e){ var b=e.target.closest(‘button’); if(!b)return; modality=b.dataset.m; [].slice.call(this.children).forEach(function(x){x.classList.toggle(‘sel’,x===b);}); metaEl.innerHTML=’Modality: ‘+(modality===’mri’?’MRI’:’CT’)+’. Content sample ratio: ‘+(ctxRatio[modality]*100)+’%. Leaflet release: 20%. Patches are 4×16×16 voxels.’; if(stage>=1) render(); }); // resize automatically: send your own offsetHeight (+40) to parent function postHeight(){ try { var h=document.body.offsetHeight+40; parent.postMessage({type:’neurovfm-voljepa-height’,height:h},’*’); }catch(e){} } window.addEventListener(‘load’,function(){ updateLoss();render();postHeight();}); window.addEventListener(‘resize’,postHeight); updateLoss(); assign (); })();

admin 4 hours ago

0 0 3 minutes read