Breaking News: Grepper is joining You.com. Read the official announcement!

spacy create tokenizer

Delightful Dormouse answered on April 20, 2021 Popularity 9/10 Helpfulness 4/10

answer spacy create tokenizer

spacy create tokenizer

Comment

# Construction 1
from spacy.tokenizer import Tokenizer
from spacy.lang.en import English
nlp = English()
# Create a blank Tokenizer with just the English vocab
tokenizer = Tokenizer(nlp.vocab)

# Construction 2
from spacy.lang.en import English
nlp = English()
# Create a Tokenizer with the default settings for English
# including punctuation rules and exceptions
tokenizer = nlp.tokenizer

Popularity 9/10 Helpfulness 4/10 Language python

Source: spacy.io

Tags: python spacy

Link to this answer
Share Copy Link

Contributed on Apr 20 2021

Delightful Dormouse

0 Answers Avg Quality 2/10

Continue with Google

By continuing, I agree that I have read and agree to Greppers's Terms of Service and Privacy Policy.

spacy create tokenizer

Contents

More Related Answers

spacy create tokenizer

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.