Model Track Record
Every prediction BarnSignal makes is logged, timestamped, and scored against actual auction results. No cherry-picking, no hidden misses. This page shows the full record.
Backtest Results
Walk-forward validation, scikit-learn, 9 barns, 3+ years of dataBefore going live, we tested 6 model configurations using walk-forward backtesting on historical USDA auction data. Each prediction was made using only data available at the time — no future leakage.
| Configuration | Features | Labels | Overall | Cattle | Hi-Conf (≥65%) | Very-Hi (≥75%) | vs Baseline |
|---|---|---|---|---|---|---|---|
| A: Random Forest (baseline) | 21 | 3-class | 39.4% | 39.4% | 46.8% | 51.3% | — |
| B: GradientBoosting | 21 | 3-class | 39.3% | 39.3% | 40.6% | 42.0% | −0.1% |
| C: GradientBoosting Binary | 21 | 2-class | 55.3% | 55.3% | 56.3% | 56.3% | +15.9% |
| D: GradientBoosting + Cultural | 33 | 3-class | 41.1% | 41.1% | 41.9% | 42.9% | +1.7% |
| E: GB Binary + Cultural | 33 | 2-class | 55.6% | 55.6% | 56.6% | 57.2% | +16.2% |
| 🏆 F: GB Binary + Cultural + Drought | 36 | 2-class | 56.2% | 56.2% | 56.7% | 56.2% | +16.9% |
The biggest single improvement (+16pp) came from switching to binary up/down labels, eliminating ambiguous “flat” predictions. Cultural calendar and drought features added another +1pp. Production model uses Random Forest Binary + Cultural Calendar (JS compatible).
What Drives the Predictions
Feature importance from best backtest config (Config F)| # | Feature | Importance | Weight |
|---|---|---|---|
| 1 | CME Basis Change | 14.2% | |
| 2 | Receipt Change % | 9.6% | |
| 3 | Drought Severity Index | 5.8% | |
| 4 | CME Futures Momentum | 5.2% | |
| 5 | CME Basis Level | 5.0% | |
| 6 | Year-over-Year Receipts | 4.9% | |
| 7 | High/Low Price Ratio | 4.8% | |
| 8 | CME Feeder/Live Spread | 4.5% | |
| 9 | Price Volatility | 3.9% | |
| 10 | Receipt Trend | 3.9% |
Methodology
How it works📊 Walk-Forward Backtesting
Each prediction uses only data available at the time. The model trains on past weeks and predicts the next auction — no peeking at future data. This mimics real-world deployment.
🌳 Random Forest + Cultural Calendar
Production runs a Random Forest classifier with 33 features: price momentum, CME futures basis, supply dynamics, and 12 Lancaster County cultural calendar features (Amish wedding season, mud sales, Farm Show, harvest, hunting).
↕ Binary Up/Down Labels
Predictions call whether next week’s average price will be higher or lower. Ambiguous near-zero moves (<$0.50/cwt) are excluded from training, which boosted accuracy by 16 percentage points.
📝 Full Transparency
Every prediction is logged to Redis with a timestamp, target date, and confidence score. When actual data arrives, it’s automatically scored. No predictions are ever deleted or hidden.
Live Resolution Log
Predictions scored against actual auction data⏳ Awaiting First Resolutions
50 predictions are pending against upcoming auctions (2023-02-14 – 2026-04-10). Once auction data comes in, predictions are automatically scored and the live accuracy record begins.
Pending Predictions
50 predictions awaiting resolution| Barn | Category | Current Price | Call | Confidence | Status | Target Date | ||
|---|---|---|---|---|---|---|---|---|
| ⏳ | Kirkwood (Tue) | Orchard/Timothy Grass - Good | $165.00 | ↓ down | 40% | Awaiting 2023-02-14 | 2023-02-14 | |
| ⏳ | Kirkwood (Tue) | Orchard Grass - Good | $165.00 | ↓ down | 40% | Awaiting 2023-02-14 | 2023-02-14 | |
| ⏳ | Kirkwood (Tue) | Alfalfa/Grass Mix - Good | $278.75 | ↓ down | 40% | Awaiting 2023-02-14 | 2023-02-14 | |
| ⏳ | Kirkwood (Tue) | Alfalfa - Good | $230.00 | ↑ up | 34% | Awaiting 2023-02-14 | 2023-02-14 | |
| ⏳ | Four States (MD) | DAIRY COWS - Lean 85-90% | $118.66 | ↓ down | 67% | Awaiting 2025-11-27 | 2025-11-27 | |
| ⏳ | Four States (MD) | DAIRY COWS - Breaker 75-80% | $136.40 | ↓ down | 67% | Awaiting 2025-11-27 | 2025-11-27 | |
| ⏳ | Four States (MD) | DAIRY COWS - Boner 80-85% | $126.96 | ↓ down | 67% | Awaiting 2025-11-27 | 2025-11-27 | |
| ⏳ | Four States (MD) | COWS - Breaker 75-80% | $159.30 | ↓ down | 67% | Awaiting 2025-11-27 | 2025-11-27 | |
| ⏳ | Four States (MD) | COWS - Boner 80-85% | $143.29 | ↓ down | 67% | Awaiting 2025-11-27 | 2025-11-27 | |
| ⏳ | Greencastle (Thu) | COWS - Breaker 75-80% | $99.44 | ↓ down | 90% | Awaiting 2026-02-06 | 2026-02-06 | |
| ⏳ | Greencastle (Thu) | COWS - Boner 80-85% | $85.44 | ↓ down | 87% | Awaiting 2026-02-06 | 2026-02-06 | |
| ⏳ | Buckhannon (WV) | COWS - Boner 80-85% | $159.40 | ↑ up | 90% | Awaiting 2026-04-01 | 2026-04-01 | |
| ⏳ | New Holland (Mon) | DAIRY COWS - Lean 85-90% | $151.04 | ↑ up | 70% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | New Holland (Mon) | DAIRY COWS - Breaker 75-80% | $195.41 | ↑ up | 70% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | New Holland (Mon) | DAIRY COWS - Boner 80-85% | $171.59 | ↑ up | 70% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Wolgemuth NH (Mon) | Wheat Straw | $191.01 | ↓ down | 85% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Wolgemuth NH (Mon) | Orchard/Timothy Grass - Good | $260.00 | ↑ up | 87% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Wolgemuth NH (Mon) | Grass - Good | $220.00 | ↓ down | 88% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Wolgemuth NH (Mon) | Corn Stalk | $142.56 | ↓ down | 90% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Wolgemuth NH (Mon) | Alfalfa/Grass Mix - Premium | $329.97 | ↓ down | 70% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Wolgemuth NH (Mon) | Alfalfa/Grass Mix - Good | $260.00 | ↑ up | 76% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Wolgemuth NH (Mon) | Alfalfa - Good | $280.00 | ↑ up | 76% | Awaiting 2026-04-06 | 2026-04-06 | |
| ⏳ | Greencastle (Mon) | DAIRY COWS - Lean 85-90% | $142.87 | ↓ down | 70% | Awaiting 2026-04-07 | 2026-04-07 | |
| ⏳ | Greencastle (Mon) | DAIRY COWS - Breaker 75-80% | $168.84 | ↓ down | 70% | Awaiting 2026-04-07 | 2026-04-07 | |
| ⏳ | Greencastle (Mon) | DAIRY COWS - Boner 80-85% | $154.35 | ↓ down | 70% | Awaiting 2026-04-07 | 2026-04-07 | |
| ⏳ | Ripley (WV) | COWS - Breaker 75-80% | $169.00 | ↑ up | 62% | Awaiting 2026-04-07 | 2026-04-07 | |
| ⏳ | Middleburg | DAIRY COWS - Lean 85-90% | $143.01 | ↓ down | 64% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Middleburg | DAIRY COWS - Breaker 75-80% | $173.98 | ↓ down | 64% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Middleburg | DAIRY COWS - Boner 80-85% | $158.07 | ↓ down | 82% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Middleburg | COWS - Boner 80-85% | $168.87 | ↑ up | 63% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Wheat Straw | $220.00 | ↑ up | 79% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Orchard/Timothy Grass - Good | $276.29 | ↑ up | 73% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Orchard Grass - Good | $203.43 | ↓ down | 73% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Grass - Premium | $330.00 | ↑ up | 90% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Grass - Good | $220.35 | ↓ down | 73% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Corn Stalk | $163.65 | ↑ up | 61% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Alfalfa/Grass Mix - Premium | $322.51 | ↓ down | 73% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Alfalfa/Grass Mix - Good | $218.47 | ↓ down | 73% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | Wolgemuth (Wed) | Alfalfa - Good | $247.69 | ↓ down | 88% | Awaiting 2026-04-08 | 2026-04-08 | |
| ⏳ | New Holland (Thu) | DAIRY COWS - Lean 85-90% | $152.60 | ↑ up | 90% | Awaiting 2026-04-09 | 2026-04-09 | |
| ⏳ | New Holland (Thu) | DAIRY COWS - Breaker 75-80% | $182.58 | ↑ up | 90% | Awaiting 2026-04-09 | 2026-04-09 | |
| ⏳ | New Holland (Thu) | DAIRY COWS - Boner 80-85% | $173.39 | ↑ up | 90% | Awaiting 2026-04-09 | 2026-04-09 | |
| ⏳ | New Holland (Thu) | COWS - Breaker 75-80% | $173.86 | ↑ up | 61% | Awaiting 2026-04-09 | 2026-04-09 | |
| ⏳ | New Holland (Thu) | COWS - Boner 80-85% | $179.47 | ↑ up | 87% | Awaiting 2026-04-09 | 2026-04-09 | |
| ⏳ | Canandaigua (NY) | STEERS - Choice and Prime 3-4 | $238.14 | ↑ up | 52% | Awaiting 2026-04-10 | 2026-04-10 | |
| ⏳ | Canandaigua (NY) | STEERS - Choice 2-3 | $227.07 | ↑ up | 52% | Awaiting 2026-04-10 | 2026-04-10 | |
| ⏳ | Canandaigua (NY) | HEIFERS - Choice and Prime 3-4 | $243.12 | ↑ up | 73% | Awaiting 2026-04-10 | 2026-04-10 | |
| ⏳ | Canandaigua (NY) | HEIFERS - Choice 2-3 | $229.68 | ↑ up | 52% | Awaiting 2026-04-10 | 2026-04-10 | |
| ⏳ | Canandaigua (NY) | COWS - Breaker 75-80% | $184.00 | ↑ up | 90% | Awaiting 2026-04-10 | 2026-04-10 | |
| ⏳ | Canandaigua (NY) | COWS - Boner 80-85% | $165.00 | ↑ up | 61% | Awaiting 2026-04-10 | 2026-04-10 | |