How would you go about appending a list to a new file?

Question

In the Python File I/O workshop I'm learning how to write create files and read and write from them. So I was wondering, instead of just writing a string, could you continuously append new items to a list and then each item show up in that file as you continue to run the code?

Answer 1 · 2020-11-16T21:58:32Z

November 16, 2020 9:58pm

Sure! I like how you are thinking about this. You are imagining "persistent objects" which is a pretty cool concept. My interest in this extends to use of NoSQL databases like MongoDB which allow you to save and query JSON objects. Very cool database concept.

Anyway...

The one thing that can be tricky here is whether or not you want to work with just flat lists of strings (easy) or if you want to save an actual generalized Python list that could contain objects, dictionaries, lists of list, etc. But that can be done too!

If you want a somewhat general-purpose disk-persistent list, you will want to investigate Python Pickle library. It is not perfect but pretty strong in its ability to dump objects to disk and reload them while retaining (most) of the data type capabilities.

Below is an lightweight implementation of a Persistent List object that is derived from Python's list object. Currently, the only way to add data to a list is append(), but using append as a model, you could easily override other list functions like insert(), extend(), pop(), sort(), and __add__().

One thing to be aware of is that when we store Pickled files, these are BINARY and not TEXT files. There are some other limitations too-- such as pickled objects won't necessarily load into other versions of Python so this is NOT portable!

# persistent_list.py
class PersistentList(list):
    import os, pickle
    def __init__(self, filename, *args, **kwargs):
        # overide the __init__ method of list
        self.filename = filename
        if os.path.exists(filename):
            data = self._load_data()
        else:
            data = list()
        super().__init__(data)

    def append(self, *args, **kwargs):
        # override the append method.
        # first call the list's append in the superclass
        super().append(*args, **kwargs)
        # store the data!
        self._store_data()

    def save(self, filename=None):
        # PUBLIC save, can override the filename
        self._store_data(filename)

    def delete(self):
        # PUBLIC clear/delete persistent storage, but not existing data
        if os.path.exists(self.filename):
            os.remove(self.filename)

    def _store_data(self, filename=None):
        # PRIVATE save as a pickle list serialization
        # Note, you can override the filename if needed
        if filename is None:
            filename = self.filename
        data = list(self)
        with open(filename, 'wb') as foutput:
            pickle.dump(data, foutput)

    def _load_data(self, filename=None):
        # load the data, you can override the filename of the class
        if filename is None:
            filename = self.filename
        # PRIVATE load data from pickle serialized file
        with open(filename, 'rb') as finput:
            data = pickle.load(finput)
        return data

Usage-- note every time you rerun this program it will continue to add more data to the persistent list!

from persistent_list import PersistentList

x = PersistentList('my_persistent_list.pkl')
x.append('I am a persistent list')
x.append({'name':'Guido Van Rostrum','age':55})
print(x)

Welcome to the Treehouse Community

Looking to learn something new?

jayda hendrickson

jayda hendrickson

How would you go about appending a list to a new file?

1 Answer

Jeff Muday

Jeff Muday