{"id":405,"date":"2026-02-16T13:33:27","date_gmt":"2026-02-16T13:33:27","guid":{"rendered":"https:\/\/infivit.com\/blog\/?p=405"},"modified":"2026-02-16T13:33:30","modified_gmt":"2026-02-16T13:33:30","slug":"mlops-practices","status":"publish","type":"post","link":"https:\/\/infivit.com\/blog\/mlops-practices\/","title":{"rendered":"Best MLOps Practices: A Real-World Case Study"},"content":{"rendered":"\n<p class=\"has-text-align-center has-contrast-color has-base-2-background-color has-text-color has-background has-link-color wp-elements-e361278fc22fa9df225fb5eaac761f4a\" style=\"font-size:15px\"><em>How we at <a href=\"https:\/\/infivit.com\/\">Infivit<\/a> transformed chaotic ML experiments into a production-ready system using modern <strong>MLOps<\/strong> practices<\/em><\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-full\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1246\" height=\"634\" data-id=\"433\" src=\"https:\/\/infivit.com\/blog\/wp-content\/uploads\/Convert-to-GIF-project.gif\" alt=\"\" class=\"wp-image-433\"\/><\/figure>\n<\/figure>\n\n\n\n<p style=\"font-size:28px\"><strong>The Hidden Crisis in Machine Learning Teams<\/strong><\/p>\n\n\n\n<p class=\"has-text-align-left\"><strong>Machine learning<\/strong> was supposed to make our lives easier. The promise was simply to build smart models, deploy them, and watch as they solve complex problems automatically. But if you&#8217;ve worked on a real ML project, you know the reality is far messier. Most ML teams today face a critical problem that has nothing to do with algorithms or model architecture. It&#8217;s not about whether to use XGBoost or Neural Networks. The real problem is <strong>memory loss<\/strong>\u2014their ML workflow has no ability to remember what worked, what failed, and why.<\/p>\n\n\n\n<p class=\"has-text-align-center has-base-2-background-color has-background\"><em>According to Google&#8217;s research on ML technical debt, only 5-10% of a real-world ML system consists of ML code. The remaining 90-95% is infrastructure, data pipelines, and operational systems.<\/em><\/p>\n\n\n\n<div class=\"wp-block-rank-math-toc-block\" id=\"rank-math-toc\"><h2>Overview<\/h2><nav><ul><li class=\"\"><a href=\"#ml-ops-practices-and-the-sentiment-analysis-project\">MLOps Practices and The Sentiment Analysis Project<\/a><\/li><li class=\"\"><a href=\"#the-root-cause-no-system-just-hope\">The Root Cause: No System, Just Hope<\/a><\/li><li class=\"\"><a href=\"#the-five-pillars-of-ml-ops\">The Five Pillars of MLOps<\/a><\/li><li class=\"\"><a href=\"#when-m-lflow-wasnt-enough\">When MLflow Wasn&#8217;t Enough<\/a><\/li><li class=\"\"><a href=\"#the-incident-that-changed-everything\">The Incident That Changed Everything<\/a><\/li><li class=\"\"><a href=\"#what-dvc-actually-does-and-why-it-matters\">What DVC Actually Does (And Why It Matters)<\/a><\/li><li class=\"\"><a href=\"#1-data-versioning-with-dvc\">1. Data Versioning with DVC<\/a><\/li><li class=\"\"><a href=\"#immediate-benefits-we-noticed\">Immediate Benefits We Noticed<\/a><ul><\/ul><\/li><\/ul><\/nav><\/div>\n\n\n\n<h3 class=\"wp-block-heading has-text-align-left\" id=\"ml-ops-practices-and-the-sentiment-analysis-project\" style=\"font-size:28px\"><strong>MLOps Practices and The Sentiment Analysis Project<\/strong><\/h3>\n\n\n\n<p class=\"has-text-align-left\">Six months ago, our team started what seemed like a straightforward project as sentiment analysis on YouTube comments. The goal was to classify comments as positive, negative, or neutral\u2014classic NLP work that any competent data scientist could handle.<\/p>\n\n\n\n<p>We followed the standard playbook:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Collect YouTube comments via API<\/li>\n\n\n\n<li>Clean and preprocess text in Jupyter notebooks<\/li>\n\n\n\n<li>Train various models (Logistic Regression, SVM, BERT)<\/li>\n\n\n\n<li>Save the best model as a .pkl file<\/li>\n\n\n\n<li>Deploy to production<\/li>\n<\/ol>\n\n\n\n<p>For the first few weeks, everything seemed fine. Our model achieved 78% accuracy on the test set. We showed demos to clients. Everyone was happy.<\/p>\n\n\n\n<p class=\"has-base-2-background-color has-background\"><em>&#8220;Why did accuracy drop from 78% to 65% in production?&#8221;<\/em><\/p>\n\n\n\n<p>We didn&#8217;t know. Was it because the production data was different from our test set? Had we accidentally changed the preprocessing code? Were we even running the same model we tested?<\/p>\n\n\n\n<p class=\"has-base-2-background-color has-background\"><em>&#8220;Which dataset did you use for the model in last Tuesday&#8217;s demo?&#8221;<\/em><\/p>\n\n\n\n<p>Our datasets were saved as comments_cleaned_v2.csv, comments_cleaned_final.csv, and comments_ACTUALLY_final.csv. Nobody could remember which was which.<\/p>\n\n\n\n<p class=\"has-base-2-background-color has-background\"><em>&#8220;Can you reproduce the results from last month&#8217;s report?&#8221;<\/em><\/p>\n\n\n\n<p>Not with any confidence. We had the code in Git, but the data had changed, the preprocessing had evolved, and we couldn&#8217;t guarantee we were using the same hyperparameters.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"the-root-cause-no-system-just-hope\"><strong>The Root Cause: No System, Just Hope<\/strong><\/h3>\n\n\n\n<p>The problem wasn&#8217;t our team&#8217;s competence. Everyone knew their stuff\u2014data cleaning, feature engineering, model training, evaluation. The problem was that we had <strong>no system<\/strong> for tracking our work.<\/p>\n\n\n\n<p>Our models were saved with names like:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"has-contrast-color has-base-2-background-color has-text-color has-background has-link-color wp-elements-d60087ceea13d464432dd36b09b0bb09\" style=\"letter-spacing:1px\">sentiment_model_final.pkl<\/li>\n\n\n\n<li class=\"has-contrast-color has-base-2-background-color has-text-color has-background has-link-color wp-elements-338908daa348893b7ec49420d4112ebf\" style=\"letter-spacing:1px\">sentiment_model_final_v2.pkl<\/li>\n\n\n\n<li class=\"has-contrast-color has-base-2-background-color has-text-color has-background has-link-color wp-elements-b2cfd687dd861a5c004bff51e1074daa\" style=\"letter-spacing:1px\">sentiment_model_REAL_final.pkl<\/li>\n\n\n\n<li class=\"has-contrast-color has-base-2-background-color has-text-color has-background has-link-color wp-elements-a670f04052a48c2f2d809a99f6e00e91\" style=\"letter-spacing:1px\">sentiment_model_use_this_one.pkl<\/li>\n<\/ul>\n\n\n\n<p>When results changed, we had no way to know if it was because of:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"has-base-2-background-color has-background\" style=\"letter-spacing:1px\">New data added to the training set<\/li>\n\n\n\n<li class=\"has-base-2-background-color has-background\" style=\"letter-spacing:1px\">Different text preprocessing steps<\/li>\n\n\n\n<li class=\"has-base-2-background-color has-background\" style=\"letter-spacing:1px\">Changed hyperparameters<\/li>\n\n\n\n<li class=\"has-base-2-background-color has-background\" style=\"letter-spacing:1px\">A bug in the code<\/li>\n<\/ul>\n\n\n\n<p class=\"has-text-align-left\">This is exactly why <strong>MLOps<\/strong> exists. Not to add complexity, but to add <strong>memory and structure<\/strong> to ML workflows.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"has-text-align-left\" style=\"font-size:28px;font-style:normal;font-weight:500;text-transform:capitalize\">Understanding MLOps workflow:<\/p>\n\n\n\n<p><strong>MLOps<\/strong> stands for <strong>Machine Learning Operations<\/strong>. At its core, it&#8217;s about bringing software engineering discipline to machine learning projects.In traditional software development, code is relatively static. Once you deploy a web application, the code doesn&#8217;t change unless you push an update. But in ML, <strong>data is constantly changing<\/strong>, which means models must continuously learn and adapt. This fundamental difference makes <strong>MLOps<\/strong> a distinct discipline.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"the-five-pillars-of-ml-ops\">The Five Pillars of MLOps<\/h3>\n\n\n\n<p>After months of research and implementation, we identified five essential pillars that form a complete MLOps system:<\/p>\n\n\n\n<p><strong>1. Version Control for Everything<\/strong><\/p>\n\n\n\n<p>Not just code, but:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Dataset versions (which data was used)<\/li>\n\n\n\n<li>Model versions (which architecture and weights)<\/li>\n\n\n\n<li>Configuration versions (hyperparameters, preprocessing settings)<\/li>\n\n\n\n<li>Environment versions (library versions, dependencies)<\/li>\n<\/ul>\n\n\n\n<p><strong>2. Automated Pipelines<\/strong><\/p>\n\n\n\n<p>Manual execution is error-prone and slow. Automated pipelines ensure:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Data loading and preprocessing run consistently<\/li>\n\n\n\n<li>Model training uses the correct parameters<\/li>\n\n\n\n<li>Evaluation metrics are calculated automatically<\/li>\n\n\n\n<li>Deployment happens smoothly without manual intervention<\/li>\n<\/ul>\n\n\n\n<p><strong>3. Experiment Tracking<\/strong><\/p>\n\n\n\n<p>Every training run is logged with complete metadata:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Performance metrics (accuracy, precision, recall, F1)<\/li>\n\n\n\n<li>Hyperparameters used<\/li>\n\n\n\n<li>Training time and computational resources<\/li>\n\n\n\n<li>Dataset identifier<\/li>\n<\/ul>\n\n\n\n<p><strong>4. Deployment Infrastructure<\/strong><\/p>\n\n\n\n<p>Models need to run somewhere accessible:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Cloud deployment (AWS, GCP, Azure)<\/li>\n\n\n\n<li>API endpoints for real-time inference<\/li>\n\n\n\n<li>Batch processing capabilities<\/li>\n\n\n\n<li>Containerization (Docker\/Kubernetes)<\/li>\n<\/ul>\n\n\n\n<p><strong>5. Continuous Monitoring<\/strong><\/p>\n\n\n\n<p>Production models need constant attention:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Performance metrics tracking<\/li>\n\n\n\n<li>Data drift detection<\/li>\n\n\n\n<li>Model drift detection<\/li>\n\n\n\n<li>Automated alerting when issues arise<\/li>\n<\/ul>\n\n\n\n<p><strong>MLOps <\/strong>is not about having the fanciest tools. It&#8217;s about having a system that prevents you from losing track of your work.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"when-m-lflow-wasnt-enough\">When MLflow Wasn&#8217;t Enough<\/h3>\n\n\n\n<p>After learning about <strong>MLOps <\/strong>principles, we implemented MLflow for experiment tracking. This solved part of our problem\u2014we could now see what <strong>hyperparameters <\/strong>we used and what metrics we achieved for each training run.<\/p>\n\n\n\n<p>But we still had a massive gap: <strong>data versioning<\/strong>.<\/p>\n\n\n\n<p><strong>MLflow <\/strong>could tell us we achieved 82% accuracy with C=1.0 and max_iter=1000, but it couldn&#8217;t tell us which version of the cleaned dataset we used. And that turned out to be critical.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"the-incident-that-changed-everything\">The Incident That Changed Everything<\/h3>\n\n\n\n<p>One Monday morning, a team member accidentally overwrote processed_data.csv with a new version that used different preprocessing parameters. The file size changed. The number of rows changed. But nobody noticed immediately.<\/p>\n\n\n\n<p>By Wednesday, when we tried to reproduce results from the previous week&#8217;s demo, we couldn&#8217;t. The data was different, and we had <strong>no way to get the old version back<\/strong>.<\/p>\n\n\n\n<p>We had the preprocessing code in Git, but:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The preprocessing parameters had changed<\/li>\n\n\n\n<li>The raw data had been updated with new comments<\/li>\n\n\n\n<li>We couldn&#8217;t remember the exact state of everything from last week<\/li>\n<\/ul>\n\n\n\n<p>That&#8217;s when we finally admitted we needed proper data versioning.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"what-dvc-actually-does-and-why-it-matters\">What DVC Actually Does (And Why It Matters)<\/h3>\n\n\n\n<p><strong>DVC (Data Version Control)<\/strong> is like Git for data and <strong>ML pipelines<\/strong>. DVC<strong> <\/strong>tracks changes to datasets without storing the actual data in Git. Instead, it stores small metadata files (.dvc files) in Git and keeps the actual data in remote storage (S3, GCS, Azure Blob, or even a local directory).<\/p>\n\n\n\n<p>DVC lets you define your <strong>ML pipeline<\/strong> as a series of stages, where each stage explicitly declares its dependencies (input files, parameters) and outputs. When you change a parameter, <strong>DVC <\/strong>automatically figures out which stages need to rerun.  With DVC, reproducing results from any point in history becomes trivial: check out the Git commit, run dvc pull to fetch the exact data versions, and run dvc repro. Done.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full wp-duotone-grayscale\"><img decoding=\"async\" width=\"1862\" height=\"786\" src=\"https:\/\/infivit.com\/blog\/wp-content\/uploads\/Screenshot-2026-02-02-125037.png\" alt=\"\" class=\"wp-image-412\" srcset=\"https:\/\/infivit.com\/blog\/wp-content\/uploads\/Screenshot-2026-02-02-125037.png 1862w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/Screenshot-2026-02-02-125037-300x127.png 300w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/Screenshot-2026-02-02-125037-1024x432.png 1024w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/Screenshot-2026-02-02-125037-768x324.png 768w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/Screenshot-2026-02-02-125037-1536x648.png 1536w\" sizes=\"(max-width: 1862px) 100vw, 1862px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"1-data-versioning-with-dvc\">1. Data Versioning with DVC<\/h3>\n\n\n\n<p>You start by <strong>data<\/strong> <strong>versioning <\/strong>your dataset using <a href=\"https:\/\/doc.dvc.org\/\" target=\"_blank\" rel=\"noopener\">DVC<\/a> to ensure reproducibility.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Initialize DVC in your project\ndvc init\n\n# Add raw data to DVC tracking\ndvc add data\/raw\/youtube_comments.csv\n\n# Commit changes to Git\ngit add data\/raw\/youtube_comments.csv.dvc .gitignore\ngit commit -m \"Add raw YouTube comments dataset\"\n\n# Push data to remote storage (e.g., AWS S3)\ndvc remote add -d myremote s3:\/\/mybucket\/dvcstore\ndvc push\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"2-training-and-experiment-tracking-with-m-lflow\">2. Training and Experiment Tracking with MLflow<\/h2>\n\n\n\n<p>Use <strong><a href=\"https:\/\/mlflow.org\/docs\/latest\/ml\/\" data-type=\"link\" data-id=\"https:\/\/mlflow.org\/docs\/latest\/ml\/\" target=\"_blank\" rel=\"noopener\">MLflow<\/a> <\/strong>to track your experiments, including parameters, metrics, and models.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># train.py\nimport mlflow\nimport mlflow.sklearn\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import accuracy_score\nimport pandas as pd\n\n# Load dataset (versioned by DVC)\ndata = pd.read_csv(\"data\/processed\/train.csv\")\nX = data.drop(\"sentiment\", axis=1)\ny = data&#91;\"sentiment\"]\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\n# Set MLflow experiment\nmlflow.set_experiment(\"sentiment-analysis\")\n\nwith mlflow.start_run():\n    # Define hyperparameters\n    C = 1.0\n    max_iter = 1000\n\n    # Log parameters\n    mlflow.log_param(\"C\", C)\n    mlflow.log_param(\"max_iter\", max_iter)\n\n    # Train model\n    model = LogisticRegression(C=C, max_iter=max_iter)\n    model.fit(X_train, y_train)\n\n    # Predict and evaluate\n    preds = model.predict(X_test)\n    accuracy = accuracy_score(y_test, preds)\n\n    # Log metrics\n    mlflow.log_metric(\"accuracy\", accuracy)\n\n    # Log model\n    mlflow.sklearn.log_model(model, \"model\")\n\n    print(f\"Logged model with accuracy: {accuracy}\")\n<\/code><\/pre>\n\n\n\n<p>Run this script after preparing your data, and MLflow will track your runs.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"3-automating-pipelines-with-dvc\">3. Automating Pipelines with DVC<\/h2>\n\n\n\n<p>Define your pipeline stages in <code>dvc.yaml<\/code> to automate preprocessing, training, and evaluation.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>stages:\n  preprocess:\n    cmd: python src\/preprocess.py\n    deps:\n      - data\/raw\/youtube_comments.csv\n      - src\/preprocess.py\n    outs:\n      - data\/processed\/train.csv\n\n  train:\n    cmd: python train.py\n    deps:\n      - data\/processed\/train.csv\n      - train.py\n    outs:\n      - model.pkl\n<\/code><\/pre>\n\n\n\n<p>Run the pipeline with:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>dvc repro\n<\/code><\/pre>\n\n\n\n<p>DVC will rerun only the necessary stages when inputs or parameters change.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"4-model-deployment-as-an-api\">4. Model Deployment as an API<\/h2>\n\n\n\n<p>Deploy the trained model as a REST API (e.g., using FastAPI or Azure Functions) so the Chrome extension can call it.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># api.py (using FastAPI)\nfrom fastapi import FastAPI\nfrom pydantic import BaseModel\nimport mlflow.sklearn\n\napp = FastAPI()\n\n# Load model from MLflow\nmodel = mlflow.sklearn.load_model(\"runs:\/&lt;run_id&gt;\/model\")\n\nclass TextRequest(BaseModel):\n    text: str\n\n@app.post(\"\/analyze\")\ndef analyze_sentiment(request: TextRequest):\n    # Preprocess input text (simplified)\n    features = preprocess_text(request.text)  # Your preprocessing function\n    prediction = model.predict(&#91;features])\n    sentiment = prediction&#91;0]\n    return {\"sentiment\": sentiment}\n<\/code><\/pre>\n\n\n\n<p>Deploy this API to a cloud service and get the endpoint URL.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"5-chrome-extension-to-use-the-api\">5. Chrome Extension to Use the API<\/h2>\n\n\n\n<p><strong>Manifest (manifest.json):<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>{\n  \"manifest_version\": 3,\n  \"name\": \"Sentiment Analyzer\",\n  \"version\": \"1.0\",\n  \"permissions\": &#91;\"activeTab\", \"scripting\"],\n  \"action\": {\n    \"default_popup\": \"popup.html\"\n  },\n  \"background\": {\n    \"service_worker\": \"background.js\"\n  }\n}\n<\/code><\/pre>\n\n\n\n<p><strong>Popup HTML (popup.html):<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;!DOCTYPE html&gt;\n&lt;html&gt;\n  &lt;head&gt;&lt;title&gt;Sentiment Analyzer&lt;\/title&gt;&lt;\/head&gt;\n  &lt;body&gt;\n    &lt;button id=\"analyzeBtn\"&gt;Analyze Selected Text&lt;\/button&gt;\n    &lt;div id=\"result\"&gt;&lt;\/div&gt;\n    &lt;script src=\"popup.js\"&gt;&lt;\/script&gt;\n  &lt;\/body&gt;\n&lt;\/html&gt;\n<\/code><\/pre>\n\n\n\n<p><strong>Popup JS (popup.js):<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>document.getElementById(\"analyzeBtn\").addEventListener(\"click\", () =&gt; {\n  chrome.tabs.query({ active: true, currentWindow: true }, (tabs) =&gt; {\n    chrome.scripting.executeScript({\n      target: { tabId: tabs&#91;0].id },\n      function: () =&gt; window.getSelection().toString()\n    }, (selection) =&gt; {\n      const text = selection&#91;0].result;\n      if (!text) {\n        alert(\"Please select some text on the page.\");\n        return;\n      }\n      fetch(\"https:\/\/your-api-url\/analyze\", {\n        method: \"POST\",\n        headers: { \"Content-Type\": \"application\/json\" },\n        body: JSON.stringify({ text: text })\n      })\n      .then(res =&gt; res.json())\n      .then(data =&gt; {\n        document.getElementById(\"result\").textContent = `Sentiment: ${data.sentiment}`;\n      })\n      .catch(err =&gt; {\n        document.getElementById(\"result\").textContent = \"Error analyzing sentiment.\";\n      });\n    });\n  });\n});\n<\/code><\/pre>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"6-monitoring-and-retraining\">6. Monitoring and Retraining<\/h2>\n\n\n\n<p>Set up monitoring on your API to track usage and model performance. If accuracy drops or data drifts, trigger retraining by updating your data, running the pipeline, and redeploying the model.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img decoding=\"async\" width=\"1080\" height=\"1080\" src=\"https:\/\/infivit.com\/blog\/wp-content\/uploads\/photo-collage.png.png\" alt=\"\" class=\"wp-image-413\" style=\"width:601px;height:auto\" srcset=\"https:\/\/infivit.com\/blog\/wp-content\/uploads\/photo-collage.png.png 1080w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/photo-collage.png-300x300.png 300w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/photo-collage.png-1024x1024.png 1024w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/photo-collage.png-150x150.png 150w, https:\/\/infivit.com\/blog\/wp-content\/uploads\/photo-collage.png-768x768.png 768w\" sizes=\"(max-width: 1080px) 100vw, 1080px\" \/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"immediate-benefits-we-noticed\">Immediate Benefits We Noticed<\/h3>\n\n\n\n<p><strong>1. Experimentation Speed Increased 5x<\/strong><\/p>\n\n\n\n<p>Before <strong>MLOps<\/strong>, running a new experiment meant manually executing 5-7 scripts in the right order, each time risking a mistake. With <strong>DVC<\/strong>, we just change parameters and run one command. Team members went from running 2-3 experiments per day to 10-15.<\/p>\n\n\n\n<p><strong>2. Onboarding Time Dropped from 3 Days to 2 Hours<\/strong><\/p>\n\n\n\n<p>New team members used to spend days figuring out which notebooks to run and which datasets to use. Now they clone the repo, run dvc pull, and have everything they need. The pipeline is self-documenting.<\/p>\n\n\n\n<p><strong>3. Zero Reproduction Failures<\/strong><\/p>\n\n\n\n<p>Before, when client asked us to reproduce results from a demo or report, we succeeded maybe 60% of the time. With DVC, we have a 100% success rate. Checkout the commit, pull the data, run the pipeline. Done.<\/p>\n\n\n\n<p><strong>4. Production Confidence Skyrocketed<\/strong><\/p>\n\n\n\n<p>We know exactly which model is in production, what data it was trained on, and what metrics it achieved. When <strong>performance <\/strong>drops, we can compare current performance against historical baselines and identify the exact change that caused the issue.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"lessons-learned\">Lessons Learned<\/h4>\n\n\n\n<p>We didn&#8217;t implement every <strong>MLOps practice<\/strong> on day one. We started with <strong>experiment tracking (MLflow)<\/strong>, then added <strong>data versioning<\/strong> <strong>(DVC)<\/strong>, then automation. Build incrementally based on your biggest pain points. Most of our failures came from data problems, not code bugs. <strong>Versioning data <\/strong>is just as important as versioning code\u2014maybe more so. Every manual step is an opportunity for mistakes. <strong>Automate<\/strong> everything that you run more than once. Our dvc.yaml and params.yaml files serve as living documentation. Team members can understand the entire pipeline just by reading these two files.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p style=\"font-size:28px\">Final Thoughts<strong>:<\/strong><\/p>\n\n\n\n<p>MLOps sounds intimidating. The term conjures images of massive infrastructure, dedicated DevOps teams, and enterprise-grade tools costing thousands of dollars per month.But that&#8217;s not what MLOps is really about.<\/p>\n\n\n\n<p><strong>MLOps <\/strong>is about having a <strong>system<\/strong>. A system for tracking your work. A system for <strong>versioning your data.<\/strong> A system for<strong> automating repetitive tasks<\/strong>. A system for deploying models reliably.<\/p>\n\n\n\n<p>You can start with free, open-source tools:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"has-base-2-background-color has-background\"><strong>DVC<\/strong> for data versioning and pipelines<\/li>\n\n\n\n<li class=\"has-base-2-background-color has-background\"><strong>MLflow<\/strong> for experiment tracking<\/li>\n\n\n\n<li class=\"has-base-2-background-color has-background\"><strong>Git\/GitHub<\/strong> for code versioning<\/li>\n\n\n\n<li class=\"has-base-2-background-color has-background\"><strong>Docker<\/strong> for containerization<\/li>\n<\/ul>\n\n\n\n<p>Don&#8217;t try to build the perfect MLOps system from day one. Build it incrementally, one pain point at a time.<\/p>\n\n\n\n<p class=\"has-text-align-center has-base-background-color has-background\"><strong><em>Machine learning should make work easier, not harder. MLOps is what makes that promise real.<\/em><\/strong><\/p>\n\n\n\n<p class=\"has-medium-font-size\"><strong>Resources:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Mlflow : <a href=\"https:\/\/mlflow.org\/docs\/latest\/ml\/\" data-type=\"link\" data-id=\"https:\/\/mlflow.org\/docs\/latest\/ml\/\" target=\"_blank\" rel=\"noopener\">https:\/\/mlflow.org\/docs\/latest\/ml\/<\/a>&nbsp;<\/li>\n\n\n\n<li>DVC :&nbsp;<a href=\"https:\/\/doc.dvc.org\/\" target=\"_blank\" rel=\"noopener\">https:\/\/doc.dvc.org\/<\/a><\/li>\n<\/ul>\n\n\n","protected":false},"excerpt":{"rendered":"<p>How we at Infivit transformed chaotic ML experiments into a production-ready system using modern MLOps practices The Hidden Crisis in Machine Learning Teams Machine learning was supposed to make our lives easier. The promise was simply to build smart models, deploy them, and watch as they solve complex problems automatically. But if you&#8217;ve worked on &#8230; <a title=\"Best MLOps Practices: A Real-World Case Study\" class=\"read-more\" href=\"https:\/\/infivit.com\/blog\/mlops-practices\/\" aria-label=\"Read more about Best MLOps Practices: A Real-World Case Study\">Read more<\/a><\/p>\n","protected":false},"author":10,"featured_media":421,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[48],"tags":[43,46,44,42,45,41,47],"class_list":["post-405","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-mlops","tag-datascience","tag-dataversioning","tag-dvc","tag-machinelearning","tag-mlflow","tag-mlops","tag-mlpipelines"],"_links":{"self":[{"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/posts\/405","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/comments?post=405"}],"version-history":[{"count":52,"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/posts\/405\/revisions"}],"predecessor-version":[{"id":523,"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/posts\/405\/revisions\/523"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/media\/421"}],"wp:attachment":[{"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/media?parent=405"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/categories?post=405"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/infivit.com\/blog\/wp-json\/wp\/v2\/tags?post=405"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}