NLP Methods Explorer
Browse classical and modern NLP methods, then open interactive demos to understand how each method works step by step.
Search and filter
Narrow the catalog by method name, category, difficulty, or demo status.
Method Catalog
Showing 12 of 12 methods.
Bigram Language Model
Predicts the next word using only one previous word.
What you can inspect
- bigram counts
- transition probability
- generated sequence
Trigram Language Model
Predicts the next word using two previous words.
What you can inspect
- trigram counts
- conditional probability
- generation trace
Markov Chain Text Generator
Generates text by moving from one state to another based on learned probabilities.
What you can inspect
- state transition table
- probability graph
- generated path
TF-IDF
Measures how important a word is inside a document collection.
What you can inspect
- term frequency
- inverse document frequency
- document score table
Word2Vec
Learns word meaning by placing similar words close together in vector space.
What you can inspect
- word vectors
- nearest words
- similarity score
RNN Language Model
Processes text step by step using hidden states.
What you can inspect
- hidden state flow
- sequence prediction
- training loss
LSTM Language Model
Improves RNNs by remembering useful information over longer sequences.
What you can inspect
- memory cell
- gates
- sequence prediction
Seq2Seq with Attention
Maps one sequence to another while using attention to focus on the most relevant input tokens.
What you can inspect
- encoder states
- attention alignment
- decoded sequence
Transformer Language Model
Uses attention to learn relationships between words across longer context.
What you can inspect
- attention weights
- token embeddings
- generated output
BERT
Learns bidirectional context for language understanding tasks such as classification and question answering.
What you can inspect
- masked tokens
- context embeddings
- classification logits
RAG
Combines document search with text generation to create grounded answers.
What you can inspect
- retrieved documents
- context chunks
- generated answer
Method Comparison Summary
A concise view of where each method is useful and what to watch for.
| Method | Main Idea | Strength | Limitation | Best For |
|---|---|---|---|---|
| 4-Gram | Predict from the previous three words. | Easy to inspect and explain. | Struggles with unseen contexts. | Learning statistical text generation. |
| TF-IDF | Score words by document importance. | Fast and transparent. | Does not understand word meaning. | Search and keyword analysis. |
| Word2Vec | Represent words as vectors. | Captures similarity between words. | Needs enough training data. | Semantic comparison. |
| Transformer | Use attention across context. | Handles long-range relationships. | More complex and resource-heavy. | Modern language modeling. |
| RAG | Retrieve documents before generation. | Can ground answers in sources. | Depends on retrieval quality. | Question answering over documents. |
Recommended Learning Path
Start with simple statistical models before moving to neural and transformer-based methods.
- 1Bigram
- 2Trigram
- 34-Gram
- 4Markov Chain
- 5TF-IDF
- 6Word2Vec
- 7RNN
- 8LSTM
- 9Transformer
- 10RAG