Skip to main content

ML Components

Each machine learning algorithm consists of the following components:

  • Representation: The way the model represents the data and the relationships between features. This can include linear models, decision trees, neural networks, etc.
  • Evaluation: The method used to assess the performance of the model, such as accuracy, precision, recall, or mean squared error.
  • Optimization: The process of adjusting the model's parameters to minimize the loss function and improve performance. This can involve techniques like gradient descent, stochastic gradient descent, or more advanced optimization algorithms.

Representation​

  • Decision Trees: A tree-like model that splits the data based on feature values to make predictions.
  • Support Vector Machines (SVM): A model that finds the optimal hyperplane that separates data points of different classes.
  • Neural Networks: A model inspired by the structure of the human brain, consisting of layers of interconnected nodes (neurons) that learn to represent complex patterns in the data.
  • Graphical Models: A model that represents the probabilistic relationships between variables using a graph structure, such as Bayesian networks or Markov random fields. Also used in SLAM.
  • Rule Sets/Logic programs: A model that uses a set of rules or logical statements to make predictions or decisions based on the input data. This can include expert systems, rule-based systems, or logic programming languages.

Evaluation​

These are various metrics used to evaluate the quality of the approach in a standardized way. The choice of evaluation metric depends on the specific problem and the goals of the machine learning task.

Optimization​

Optimization is the process of adjusting the model's parameters to minimize the loss function and improve performance. This can involve techniques like

  • Combinatorial Optimization: Finding the best combination of parameters like greedy search.
  • Convex Optimization: Finding the optimal solution in a convex problem space like gradient descent.
  • Constrained Optimization: Finding the optimal solution while satisfying certain constraints like linear programming, quadratic programming, or nonlinear programming.