Kurt McKee

lessons learned in production

Articles tagged with python

Design for "and" and "or" | 12 June 2020

Remember that one time when your software requirements changed and you realized that you would have to make dramatic changes to your code? Remember how that's only happened to you once?

I've been working on the same codebase for eight years. During that time I've learned that requirements change, and I've slowly learned some design patterns that can help make code flexible enough to accommodate changing requirements.

I'd like to share some of those experiences and lessons with you.

Self-documenting code | 26 December 2019

I have to maintain code and narrative documentation for compliance reasons at work, but I don't want to separately update the documentation each time I update the code! Luckily, because I'm using Python and Sphinx, it's easy to the automatically update the docs.

I'll be in the Netherlands | 13 January 2013

Hey everybody, I'm going to be in Zaltbommel and Waardenburg, Netherlands starting February 4th for two weeks. If anyone would like to get together with me while I'm there, please let me know! I'd be delighted to meet with you, have a drink, and talk about feedparser or Python in …

pdb post-mortem debugging | 13 January 2012

I misread a recent feedparser bug report and thought that the author had mistyped a bbcode- or wiki-formatted link. Consequently, I wasted time debugging the code.

Turns out, I wasted a lot of time, because I spent more than zero time debugging the code. Yes, zero time. I was using …

Extending feedparser using dynamic inheritance | 14 December 2010

After going through many, many feedparser bugs (over 100 closed since I started working on the project, with more waiting to be reviewed!) I've seen a lot of reports asking for additional support for various namespaces. Off the top of my head the list includes GeoRSS, the iTunes Music Store …

Feedparser porting status | 23 November 2010

I spent some time this weekend working on porting feedparser to Python 3, and found that it will be difficult because there are two separate parsers included (a strict parser and a loose parser), and while each works differently, both use the same core machinery in feedparser.

With the strict …

Porting feedparser to Python 3 | 19 November 2010

Today I'm announcing I'm throwing my hat in the ring to port feedparser to Python 3. There's a ticket open regarding this at the feedparser bug tracker, but the person who's working on porting it appears to be writing Python 3 code instead of writing Python 2 code that the …

Quantifying unit test coverage | 4 July 2009

I've really been working hard to make sure that every piece of code that I commit to listparser is backed by unit tests to ensure that the code does what I expect it to do. But a while back I had the idea that it would be great to have …

Subclassing Python types | 6 March 2009

I recently found a need to subclass the builtin type unicode and add some additional properties. To instantiate I wanted to pass in a big, ugly object and get a unicode object back. After trying fruitlessly to override __init__, I finally read up on the Python data model. Turns out …

Wordpress.com and comment feeds | 2 August 2008

A few days ago I added support for tracking Wordpress.com comments, but its implementation leaves a lot to be desired, as I'm merely using comment feeds for the purpose.

The first problem is that Wordpress.com limits the number of comments in feeds to 10. Thus, it is very …

Revisiting comment scraping | 25 July 2008

Earlier this month I wrote about scraping LiveJournal comments. What was I thinking?

While I was able to account for a number of variables in the page by tweaking my XPath statements, it became obvious early on that screen scraping for comments should be a last resort. So I decided …

Scraping LiveJournal comments | 8 July 2008

As a first attempt at expanding my comment tracking software, I did a little testing in regards to scraping LiveJournal comments. Having written some uncomfortably convoluted XSL transformations in the past, I've become familiar with XPath. While BeautifulSoup has served me well in the past for quick excursions into the …

Comments elsewhere | 2 July 2008

I've been working very hard to track down and save up all of my comments on others' websites. Although there are still some comments out there, I've manually added over 300 comments, which has been a serious chore.

Tonight I slapped a little CSS together so that the comment viewing …

Python's time module | 12 June 2008

So I've been working like a fiend to import all of my comments from around the internet. It has been a herculean effort because almost everything has to be done manually. One comment here, another there, and no uniform way to extract those comments.

Two sites, however, made comment retrieval …