パンダのノートブックはこちら
qcut が新しい列として DataFrame に割り当てられた結果が表示されます。Dataframe には 2 つの列がありますが、どういうわけか qcut 出力を新しい列に割り当てると、「var」変数が該当する正しいレベルが魔法のように検出されます。他の変数はチェックされません。ここでパンダのセマンティクスは何ですか? サンプル出力は以下のとおりです。
In [2]:
from pandas import *
from statsmodels.formula.api import logit
from statsmodels.nonparametric import KDE
from patsy import dmatrix, dmatrices
In [3]:
df = read_csv('wells.dat', sep = ' ', header = 0, index_col = 0)
print df.head()
switch arsenic dist assoc educ
1 1 2.36 16.826000 0 0
2 1 0.71 47.321999 0 0
3 0 2.07 20.966999 0 10
4 1 1.15 21.486000 0 12
5 1 1.10 40.874001 1 14
In [4]:
model_form = ('switch ~ center(I(dist / 100.)) + center(arsenic) + ' +
'center(I(educ / 4.)) + ' +
'center(I(dist / 100.)) : center(arsenic) + ' +
'center(I(dist / 100.)) : center(I(educ / 4.)) + ' +
'center(arsenic) : center(I(educ / 4.))'
)
model4 = logit(model_form, df = df).fit()
In [20]:
resid_df = DataFrame({'var': df['arsenic'], 'resid': model4.resid})
resid_df[:10]
Out [20]:
resid var
1 0.842596 2.36
2 1.281417 0.71
3 -1.613751 2.07
4 0.996195 1.15
5 1.005102 1.10
6 0.592056 3.90
7 0.941372 2.97
8 0.640139 3.24
9 0.886626 3.28
10 1.130149 2.52
In [15]:
qcut(df['arsenic'], 40)
Out [15]:
Categorical: arsenic
array([(2.327, 2.47], (0.68, 0.71], (1.953, 2.07], ..., [0.51, 0.53],
(0.62, 0.64], (0.64, 0.68]], dtype=object)
Levels (40): Index([[0.51, 0.53], (0.53, 0.56], (0.56, 0.59],
(0.59, 0.62], (0.62, 0.64], (0.64, 0.68],
(0.68, 0.71], (0.71, 0.75], (0.75, 0.78],
(0.78, 0.82], (0.82, 0.86], (0.86, 0.9], (0.9, 0.95],
(0.95, 1.0065], (1.0065, 1.0513], (1.0513, 1.1],
(1.1, 1.15], (1.15, 1.2], (1.2, 1.25], (1.25, 1.3],
(1.3, 1.36], (1.36, 1.42], (1.42, 1.49],
(1.49, 1.57], (1.57, 1.66], (1.66, 1.76],
(1.76, 1.858], (1.858, 1.953], (1.953, 2.07],
(2.07, 2.2], (2.2, 2.327], (2.327, 2.47],
(2.47, 2.61], (2.61, 2.81], (2.81, 2.98],
(2.98, 3.21], (3.21, 3.42], (3.42, 3.791],
(3.791, 4.475], (4.475, 9.65]], dtype=object)
In [17]:
resid_df['bins'] = qcut(df['arsenic'], 40)
resid_df[:20]
Out [17]:
resid var bins
1 0.842596 2.36 (2.327, 2.47]
2 1.281417 0.71 (0.68, 0.71]
3 -1.613751 2.07 (1.953, 2.07]
4 0.996195 1.15 (1.1, 1.15]
5 1.005102 1.10 (1.0513, 1.1]
6 0.592056 3.90 (3.791, 4.475]
7 0.941372 2.97 (2.81, 2.98]
8 0.640139 3.24 (3.21, 3.42]
「var」の正しいビンが見つかりましたが、割り当ては「resid」に注意を払っていませんでした。