Which of the following are spark MLlib tools?

Spark MLlib provides the following tools:

• ML Algorithms: ML Algorithms form the core of MLlib.
• Featurization: Featurization includes feature extraction, transformation, dimensionality reduction and selection.
• Pipelines: Pipelines provide tools for constructing, evaluating and tuning ML Pipelines.

What is BinaryClassificationEvaluator?

Class BinaryClassificationEvaluator Evaluator for binary classification, which expects two input columns: rawPrediction and label. The rawPrediction column can be of type double (binary 0/1 prediction, or probability of label 1) or of type vector (length-2 vector of raw predictions, scores, or label probabilities).

What algorithms are in MLlib?

MLlib types, algorithms and utilities

• summary statistics. correlations.
• linear models (SVMs, logistic regression, linear regression) naive Bayes.
• alternating least squares (ALS)
• k-means. Gaussian mixture.
• singular value decomposition (SVD) principal component analysis (PCA)
• FP-growth.
• stochastic gradient descent.

What is StringIndexer Pyspark?

StringIndexer encodes a string column of labels to a column of label indices. If the input column is numeric, we cast it to string and index the string values.

What is false positive in confusion matrix?

false positives (FP): We predicted yes, but they don’t actually have the disease. (Also known as a “Type I error.”) false negatives (FN): We predicted no, but they actually do have the disease.

Is Pyspark ML same as MLlib?

spark. mllib is the first of the two Spark APIs while org.apache.spark.ml is the new API. mllib carries the original API built on top of RDDs. spark.ml contains higher-level API built on top of DataFrames for constructing ML pipelines.

What kind of data structures does Pyspark MLlib built in library support in spark?

MLlib fits into Spark’s APIs and interoperates with NumPy in Python (as of Spark 0.9) and R libraries (as of Spark 1.5). You can use any Hadoop data source (e.g. HDFS, HBase, or local files), making it easy to plug into Hadoop workflows.