Standards standards standards

I’m sitting in a bar at the moment, away from the Yellow Face. Officially I’m working on either my “practice” NaNoWriMo novel or an updated version of a superhero campaign I once ran that was pretty successful and which I’m thinking of running again. Both of which I may write about later. In practice my mind is jumping all over the place as usual, and the fact that I’m doing this on a computer increases the temptation to geek out.

My “standards” side has been thinking about writing format issues recently. I move so often from platform to platform and program to program, writing things on each and expecting to be able to put them all together, that “what do I save this as?” is a real issue.

The number one readable format out there is plain ASCII text of course. If you can get a text file onto a device, it can display and usually edit it, encoding issues aside but there are standards there too. (When I say “plain text” here, I mean ASCII that’s meant to be displayed exactly as written, rather than ASCII that contains markup commands, like LaTEX or HTML.) For writing purposes I therefore try to create everything in plain text. My current favourite document processing program, Ulysses, only allows you to write unformatted plain text, supposedly to concentrate your mind on the content rather than the formatting – its purpose is to help you organise your documents and thoughts to produce a whole. You can export in different formats but that’s something you do at the end.

When I’m just taking notes I write in plain text, usually with BBEdit on the Mac, or a program called EditPlus at work. I admit to using Stickies sometimes when I’m taking completely disposable notes, but those rarely last more than a few weeks. Before getting hold of Ulysses I would be forced to use a full word processor to produce anything of any length, and so get caught up in tinkering with fonts and paragraph styles and all of that nonsense, but Ulysses eliminates the need for that, with the consequence that I write twice as much. I have recently got caught up in writing a Perl program that translates Ulysses’ export output into a series of web pages… but I’ve finished that now, so no more excuses.

What’s really concerning me now though is blog entries. I’ve made an awful lot of these and unfortunately, I’ve gotten lazy. I like to put formatting (quotes, images etc) into the entries which should mean that they’re in straight HTML, but they’re not – because most of the blogging packages and sites offer some level of automatic translation, with text seperated by blank lines being properly tagged as paragraphs, the entries end up as a nasty combination of plain text and HTML, and what’s more, a non-standard one. Something that displays well in WordPress (which has a powerful reformatting engine that makes almost anything look okay) won’t look the same in Movable Type, or Livejournal, or Blogger, and if I save it and open it in a web browser it looks like nothing at all. And that’s not good. It’s tying me to a particular program and reducing my options. Besides, I feel dirty writing badly-formatted code. Dirty and disgusting.

I’ve been looking at using Markdown, which has the advantage that the source it uses is readable plain text on its own. Markdown is really just a series of rules for turning that plain text into HTML and the software to do that with. There is a Markdown plugin for WordPress, but it’s either on or off, you can’t select different display formats for individual posts with WordPress, unlike MT. If it’s on all my old lazy entries are unreadable if they’re not good HTML. (Markdown doesn’t seem to mess with properly-tagged HTML.)

What I really need to do is to get WordPress to translate my old entries into HTML, and then save them, so if I turn Markdown on, it doesn’t screw everything up. I don’t see any basic reason why this isn’t possible but I’m not aware of any existing mechanism to do it. I might have to write my own program. Well, I do need to get some experience using PHP with SQL. I’m sure it wouldn’t be that hard:

1. find out which function the WordPress display routine uses to translate entries;
2. write a loop to get all entries and run that function on them;
3. for each one, once you’ve run the function, instead of displaying the results, overwrite the original with the results.

Hey, I have things backed up. If you see some funny stuff going on here in the near future, it’s probably me screwing about trying to get this to work.

Oh yeah, the other way would be to hack the Markdown plugin so it only Markdowned entries past a certain date – a lot easier – but that would leave all my old entries messy and non-standard.

Edit: apparently this latter approach isn’t possible. I just had a go myself and saw some basic problems, and then found this Wiki entry. Botheration.

Edit 2: I’m so not expecting anyone to read this whole entry. If you’re reading this, leave your name, address and credit card number below and I’ll send you a donut. Did I say credit card number? It’s for, um, validation purposes.