4

だから私はテキストファイルに単語のリストを持っています。それらに対して見出し語化を実行して、同じ意味で時制が異なる単語を削除したいと考えています。try、tried などのように、これを行うと、 TypeError: unhashable type: 'list' のようなエラーが発生し続けます

    results=[]
    with open('/Users/xyz/Documents/something5.txt', 'r') as f:
       for line in f:
          results.append(line.strip().split())

    lemma= WordNetLemmatizer()

    lem=[]

    for r in results:
       lem.append(lemma.lemmatize(r))

    with open("lem.txt","w") as t:
      for item in lem:
        print>>t, item

すでにトークンになっている単語を見出し語にするにはどうすればよいですか?

4

2 に答える 2

5

メソッドWordNetLemmatizer.lemmatizeはおそらく文字列を期待していますが、文字列のリストを渡しています。これはあなたにTypeError例外を与えています。

の結果line.split()は、リストとして追加する文字列のリスト、resultsつまりリストのリストです。

使いたいresults.extend(line.strip().split())

results = []
with open('/Users/xyz/Documents/something5.txt', 'r') as f:
    for line in f:
        results.extend(line.strip().split())

lemma = WordNetLemmatizer()

lem = map(lemma.lemmatize, results)

with open("lem.txt", "w") as t:
    for item in lem:
        print >> t, item

または中間結果リストなしでリファクタリング

def words(fname):
    with open(fname, 'r') as document:
        for line in document:
            for word in line.strip().split():
                yield word

lemma = WordNetLemmatizer()
lem = map(lemma.lemmatize, words('/Users/xyz/Documents/something5.txt'))
于 2015-12-03T06:28:31.853 に答える
1 
Open a text file and and read lists as results as shown below
fo = open(filename)
results1 = fo.readlines()

results1
['I have a list of words in a text file', ' \n I want to perform lemmatization on them to remove words which have the same meaning but are in different tenses', '']

# Tokenize lists

results2 = [line.split() for line in results1]

# Remove empty lists

results2 = [ x for x in results2 if x != []]

# Lemmatize each word from a list using WordNetLemmatizer

from nltk.stem.wordnet import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()
lemma_list_of_words = []
for i in range(0, len(results2)):
     l1 = results2[i]
     l2 = ' '.join([lemmatizer.lemmatize(word) for word in l1])
     lemma_list_of_words.append(l2)
lemma_list_of_words
['I have a list of word in a text file', 'I want to perform lemmatization on them to remove word which have the same meaning but are in different tense']

Please look at the lemmatized difference between lemma_list_of_words and results1.
于 2017-10-09T08:11:49.033 に答える