November 5, 2025.

LiveScope is an internal hour-estimation tool for Control Southern graphics engineering work. It takes structured counts of Dynamos, Graphics, and Contextuals and predicts engineering hours using trained Random Forest models. The product grew from spreadsheet experiments into a Streamlit web app with prepare, predict, and training-data workflows, plus packaging for eventual server deployment behind Cloudflare Zero Trust.

The core problem was translating messy plant exports into a consistent feature space. Raw EMR Summary files did not match the training layout, so we built a prepare pipeline that filters “In Scope” rows and maps plant-specific columns into a shared LiveScope schema: Name, feature columns, and Hours. That mapping had to be sheet-aware—Dynamos, Graphics, and Contextuals each use different fields—and validated against known Emory reference workbooks. Small mismatches, such as Graphics Dynamos counting empty versus zero, showed how fragile export conventions can be and why automated conversion needed careful column resolution rather than hard-coded positions.

Two training regimes were maintained in parallel: a full “no reduction” set and a “reduced” set, producing six models total (three sheets × two datasets). Features exclude Name; Plant is treated as categorical with ordinal encoding; Complexity is coerced from text labels (Simple/Medium/Complex) into numeric levels; other columns are numeric with median imputation. Models are RandomForestRegressor pipelines tuned with RandomizedSearchCV under five-fold cross-validation, then scored on an 80/20 holdout and refit on all labeled rows for production artifacts. After feature policy changes—most notably dropping Width and Height as non-significant—the models were retrained so prediction metadata matched the new columns.

Obstacles appeared at every layer. Early Excel workbooks carried Unnamed junk columns that polluted feature sets. Mixed types and categorical Plant values broke naive numeric pipelines until preprocessing was centralized. Training under constrained environments required n_jobs=1 to avoid multiprocessing failures. Prediction needed dual outputs—Hours_Regular and Hours_Reduced—on the same sheets without requiring Hours in the input. Appending new plants meant accepting blank Hours so engineers could fill targets later, while still injecting Plant, converting Complexity, skipping Plant+Name duplicates, and avoiding Width/Height writes. Operational issues also mattered: locked Excel files blocked appends; Windows console encoding crashed on Unicode arrows; Streamlit’s first-run email prompt and oversized inline SVGs hurt the UI until logos and CSS were fixed.

On the product side, the team moved from CLI scripts and tkinter pickers to a branded Streamlit app (Prepare and Predict only; retrain kept offline). That raised deployment questions: localhost is not shareable, so the path forward became self-hosting with Cloudflare Access rather than baking login into Streamlit. Engineering discipline included pinned dependency versions so pickled models load safely, runtime packaging that ships models but excludes customer source Excel, and a laptop-to-server promotion model for code and artifacts.

Empirically, the strongest model was No Reduction Graphics (holdout R² ≈ 0.88, MAE ≈ 0.85 hours). Dynamos and Contextuals were weaker, and Reduced Contextuals degraded on holdout—evidence that sheet size, target noise, and reduction policy dominate accuracy more than algorithm novelty. Overall, LiveScope succeeded by combining careful data standardization, disciplined train/validate/test evaluation, and practical software engineering that turned plant exports into a usable estimation workflow.