python - それらにアクセスできない別のオブジェクトから1行ずつファイル/カーソルを消費する方法

Question

CSVファイルまたはDBからオブジェクト（タスク）を読み取るプログラムがあります。どちらのソースにも、使用後にリソースへのアクセスを明示的に閉じる必要があるという共通点があります。

CSVクラスとDBクラスの両方を反復可能にするアプローチに従ったので、それらを反復するとタスクが返されます。これはそれらを使用するのに便利ですが、私はそれがきれいであるとは確信していません、そして私は次の質問があります：

ファイルまたはDBにアクセスするための最良の方法は何ですか？そのためにコンストラクターを使用していますが、マルチスレッドなどについてはよくわかりません...
リソース（ファイル、カーソル）を閉じるための最良の方法は何ですか。外部オブジェクトがアクセスが完了したことを通知する必要がありますか、それともCSVまたはDBオブジェクトがファイルの最後にいることを検出して、ファイルを閉じる必要がありますか？

私はこれを正しく行っているかどうかわかりません（これはシングルランで機能しますが、これはWebサイトに接続されるため、複数のアクセス権があります）

class CSV(AbstractDAO):
    def __init__(self, sourcePath):
        self.sourcePath = sourcePath
        self.csvFile = codecs.open(sourcePath, 'rb', 'UTF-8')

    def __iter__(self):
        return self

    def next(self):
        return self._buildTaskFromLine(self.csvFile.next())

    def deleteAllTasks(self):
        pass

    def loadTask(self, taskID):
        csvFile = codecs.open(self.sourcePath, 'rb', 'UTF-8')
        for line in csvFile:
            taskValues = line.split(";")
            if taskValues[0] == unicode(taskID):
                return self._buildTaskFromLine(line)
            else:
                return None

    def saveTask(self, task):
        pass

    def loadPredecessorsID(self, task):
        csv = codecs.open(self.sourcePath, 'rb', 'UTF-8')
        for line in csv:
            taskValues = line.split(";")
            if taskValues[0] == unicode(task.id):
                return taskValues[2].split(",")
        return None

    def _buildTaskFromLine(self, line):
        taskValues = line.split(";")
        taskID = taskValues[0]
        taskName = taskValues[1]
        taskAncestors = taskValues[2]
        taskDuration = int(taskValues[3])
        return Task(taskID, taskName, taskDuration)

これがDBの実装です

class SQLite(AbstractDAO):
    def __init__(self, sourcePath):
        self.connection = sqlite3.connect(sourcePath)
        self.cursor = None

    def __iter__(self):
        self.cursor = self.connection.cursor()
        self.cursor.execute("select * from Tasks")
        return self

    def next(self):
        if self.cursor is not None:
            row = self.cursor.fetchone()
            if row is None:
                self.cursor.close()
                raise StopIteration
            else:
                return self._buildTaskFromRow(row)

    def deleteAllTasks(self):
        cursor = self.connection.cursor()
        cursor.execute("delete from Tasks")
        self.connection.commit()
        cursor.close()

    def loadTask(self, id):
        cursor = self.connection.cursor()
        param = (id,)
        cursor.execute("select * from Tasks t where t.id = ? ", param)
        taskRow = cursor.fetchone()
        task = self._buildTaskFromRow(taskRow)
        cursor.close()
        return task

    def saveTask(self, task):
        cursor = self.connection.cursor()
        param = (task.id,)
        cursor.execute("select * from Tasks t where t.id = ? ", param)
        taskRow = cursor.fetchone()
        if taskRow is None:
            param = (task.id, task.name, task.duration)
            cursor.execute("insert into Tasks values (?,?,?)", param)
            self.connection.commit()
            cursor.close()

        else:
            param = (task.id, task.name, task.duration)
            cursor.execute("update Tasks \
            set description = ?, duration = ?  \
            where id = ? ", param)
            self.connection.commit()
            cursor.close()

    def loadPredecessors(self, task):
        pass

    def _buildTaskFromRow(self, row):
        taskId = row[0]
        taskName = row[1]
        taskDuration = row[2]
        return Task(taskId, taskName, taskDuration)

最後に、上記のコードは、たとえばma TaskTreeによってこのように呼び出されます（これはすべてのタスクを保持するオブジェクトです）

def loadTreeFrom(self, source, sourcePath):
    if source not in ('CSV', 'DB'):
        raise AttributeError('Unknown source : supported sources are CSV or DB')

    dao = None
    if source == 'CSV':
        dao = CSV(sourcePath)
    elif source == "DB":
        dao = SQLite(sourcePath)

    #populate the tasks first
    for task in dao:
        self.tasks[unicode(task.id)] = task

    # then populate the dependencies
    for item in self.tasks.iteritems():
        ancestorsID = dao.loadPredecessorsID(item[1])
        self.addDependencies(item[1], ancestorsID)

score 2 · Accepted Answer

This is sort of a sideways answer to your question, but based on your description, I think you should consider making these objects into context managers. That way, instead of having your "external object notify the access is done," you can simply use a with block. When the block is entered, your context manager object's __enter__ method is called; when it is exited, your context manager object's __exit__ method is called. Here's a (very) simple example:

>>> class DummyManager(object):
...     def __enter__(self):
...         print 'entering with block!'
...         return 'foo'
...     def __exit__(self, exc_type, exc_val, exc_tb):
...         print 'exiting with block!'
...         print 'this is exception info, if an exception was raised:'
...         print exc_type, exc_val, exc_tb
... 
>>> with DummyManager() as dummy:
...     print dummy
... 
entering with block!
foo
exiting with block!
this is exception info, if an exception was raised:
None None None

This is a nice way for the iterating object to let your DB/CVS object know that it's no longer needed.

Honestly, I have no precise idea how you should handle concurrent access, etc -- since I don't know your overall design. But the __enter__ and __exit__ methods are potentially good places to handle locks, etc., if you need them.

One way to restructure your classes based on this system would be to write the methods assuming that the resource is open, instead of opening and closing it all the time. Then always refer to instances of the object within a with block; the with statement takes care of initializing and opening resources, and closing them when control leaves the block. So for example your loadTask and saveTask methods would no longer require x.open(...) and x.close() lines at the beginning and end, and the resource is open exactly as long as it is being used.

If you like, you can create public open and close methods, and then just have __enter__ and __exit__ call them. Then your user (or you) can decide whether to use a with block or to open and close the object in the classic style. Either way, the object behaves an a way analogous to a file in Python. (And did I mention that files are also context managers?)

Based on your new code, you would then call the resources something like this:

def loadTreeFrom(self, source, sourcePath):
    if source not in ('CSV', 'DB'):
        raise AttributeError('Unknown source : supported sources are CSV or DB')

    dao = None
    if source == 'CSV':
        dao = CSV            # dao just a reference to the class now
    elif source == "DB":
        dao = SQLite

    with dao() as tasks:
        #populate the tasks first
        for task in tasks:
            self.tasks[unicode(task.id)] = task

        # then populate the dependencies
        for item in self.tasks.iteritems():
            ancestorsID = dao.loadPredecessorsID(item[1])
            self.addDependencies(item[1], ancestorsID)

Now when you call loadPredecessorsID, it doesn't open and close the resource every time.

python - それらにアクセスできない別のオブジェクトから1行ずつファイル/カーソルを消費する方法

1 に答える 1

Related

Reference