AWS Certified Machine Learning - Specialty (MLS-C01) Domain 3

Modeling

Official Exam Guide: Domain 3: Modeling

Skill Builder: AWS Certified Machine Learning - Specialty Exam Prep

---

Domain Overview

Domain 3 (36% - largest domain) focuses on framing business problems as ML problems, selecting appropriate models, training models, hyperparameter optimization, and evaluating models.

---

Task 3.1: Frame business problems as ML problems

Key Concepts:

• Determine when to use/not use ML
• Supervised vs unsupervised learning
• Problem types: classification, regression, forecasting, clustering, recommendation, foundation models

Essential Documentation:

• Choose an Algorithm
• Types of ML Models

---

Task 3.2: Select appropriate model(s)

Key Algorithms:

• XGBoost, logistic regression, k-means, linear regression, decision trees, random forests
• RNN, CNN, ensemble methods, transfer learning, LLMs
• Express intuition behind models

Essential Documentation:

• XGBoost Algorithm
• Linear Learner Algorithm
• K-Means Algorithm
• Random Cut Forest
• DeepAR Forecasting Algorithm
• Image Classification Algorithm
• Object Detection Algorithm

---

Task 3.3: Train ML models

Key Concepts:

• Split data (training/validation, cross-validation)
• Optimization techniques (gradient descent, loss functions, convergence)
• Compute resources (GPU/CPU, distributed/non-distributed, Spark/non-Spark)
• Model updating (batch, real-time/online)

Essential Documentation:

• Train a Model with Amazon SageMaker
• Distributed Training
• SageMaker Data Parallelism
• SageMaker Model Parallelism

---

Task 3.4: Perform hyperparameter optimization

Key Concepts:

• Regularization (dropout, L1/L2)
• Cross-validation
• Model initialization
• Neural network architecture (layers, nodes, learning rate, activation functions)
• Tree-based models (number of trees, depth)
• Linear models (learning rate)

Essential Documentation:

• SageMaker Automatic Model Tuning
• How Hyperparameter Tuning Works

---

Task 3.5: Evaluate ML models

Key Concepts:

• Avoid overfitting/underfitting (bias-variance tradeoff)
• Metrics: AUC-ROC, accuracy, precision, recall, RMSE, F1 score
• Confusion matrices
• Offline and online evaluation (A/B testing)
• Model comparison (training time, quality, engineering costs)
• Cross-validation

Essential Documentation:

• Amazon SageMaker Model Monitor
• Monitor Models for Data and Model Quality
• SageMaker Model Dashboard

---

AWS Service FAQs

• Amazon SageMaker FAQs

---

Study Tips

1. Master algorithm selection - Classification (XGBoost, random forest), Regression (linear learner), Clustering (k-means), Forecasting (DeepAR), Computer Vision (CNN), NLP (RNN, transformers).

2. Learn evaluation metrics - Classification: accuracy, precision, recall, F1, AUC-ROC. Regression: RMSE, MAE, R². Confusion matrix interpretation.

3. Understand hyperparameter tuning - SageMaker automatic model tuning, Bayesian optimization, random search, grid search strategies.

4. Practice bias-variance tradeoff - Overfitting (high variance): regularization, more data. Underfitting (high bias): more features, complex model.

5. Study training optimization - Distributed training strategies, gradient descent variants (SGD, Adam), learning rate schedules, early stopping.

---

Note: This is Domain 3 of 4, representing 36% (largest domain) of exam content.