Space-based tokenisation can be used for most languages like (Select all that apply)

	(a)	Japanese			(b)	Chinese
	(c)	Arabic			(d)	Thai
	(e)	Cyrillic			(f)	English

Space-based tokenisation can be done with unix tools like:

	(a)	ed			(b)	tr
	(c)	cp			(d)	cut

We can remove punctuation since it holds no real meaning for machines

(a)

True

(b)

False

Byte-Pair Encoding is composed of 2 parts which are

	(a)	learner and tokeniser			(b)	segmenter and learner
	(c)	vocab corpus and tokens			(d)	segmenter and classifier

Byte-Pair Encoding often includes subwords like morphemes

(a)

True

(b)

False

Morphological parsing is

	(a)	Dividing words into the smallest subwords with meaning			(b)	Finding which words have been morphed or modified
	(c)	Identify changes that occurred to the root word			(d)	Parsing own words by morphing them into another entity or token

Stemming is a more sophisticated form of lemmatization 1

	(a)	True
	(b)	False

Reducing relational to relate via a rule that turns “ATIONAL” TO “ATE” is an example of

	(a)	Stemming			(b)	Porter Stemmer
	(c)	Lemmatisation			(d)	Regular Expressions

The most problematic symbol when segmenting sentences:

	(a)	?			(b)	!
	(c)	,			(d)	.

What can we use to deal with symbols that may mark an end of sentence or part of a word?

	(a)	Use rules or machine learning to classify
	(b)	Use word segmentation and prune symbols
	(c)	Use POS (part of speech tagging)
	(d)	Tokenise after filtering out symbols

What can we use to mitigate email scams?

	(a)	Text processing			(b)	Text classification
	(c)	Sentiment analysis			(d)	Lemmatisation

Hybrid machine translation relies on

	(a)	Text classification			(b)	Text processing
	(c)	Recurring Neural Networks			(d)	Data filtering

The first layer of Text classification is

	(a)	Classification Algorithm			(b)	Feature extract
	(c)	Vector of features			(d)	Class label
	(e)	Probability distribution

It is not possible to generate classification features by hand

	(a)	True
	(b)	False

Which of the following is not true about the BOW representation for text? (Bag of words)

	(a)	Represented by vectors or matrices of 1s and 0s
	(b)	Each word represented equally
	(c)	Context taken into consideration
	(d)	Features is the number of unique words in vocabulary

What solves the space dimensionality problem of BOW?

	(a)	NBOW			(b)	Using Ngrams
	(c)	Linearising sum			(d)	Using pre-trained word embeddings

NBOW retains the problem of the Bag of Words where all words are treated equally

(a)

True

(b)

False

Given the data below Find the probability for the classification of the following sentence “Winston Ana dark” for it being the class OW (2 non zero decimal places)

Answer: 

Given the data below Find the probability for the classification of the following sentence “Winston Ana dark” for it being the class COD (2 non zero decimal places)

Answer: