The mailman mailing list manager allows to download monthly archives in "Gzip'd Text" format. This format is not mbox but can easily be turned into it by the following simple line of sed:
sed 's/^\(From:\? .*\) \(at\|en\) /\1@/'
This makes it much easier to browse paste emails of an archive eg. using mutt or reply to past emails with a proper Message-ID as to not break threads.
This is the title of a great talk at the last chaos communication congress in berlin (27c3).
When writing my own pdf parser for a homework assignment that I put way too much ambition into, I encountered all of what is mentioned in that talk and was also able to realize how bad the situation really is. I just deleted three paragraphs of this post where I started to rant about how frickin bad the pdf format is. How unworkable it is and how literally impossible to perfectly implement. But instead, just watch the video of the talk and make sure to remember that it is even worse than Julia Wolf is able to make clear in 1h she was given.
So after I stopped myself from spreading another wave of my pdf hate over the internets lets look at the issue at hand:
This is the snippet I uncompressed from the pdf to (just by chance) find the number I was looking for. The 000000 piece was actually containing the number I needed.
6 0 obj
/DA (/Arial 14 Tf 0 g)
/Rect [ 243.249 176.784 382.489 210.297 ]
/BG [ 0.75 0.75 0.75 ]
/P 4 0 R
/N 7 0 R
So let me say: WTF? My bank not only requires me to resort to one specific pdf implementation (namely the acrobat reader by adobe) but also requires me to pay to a US based company first to have an operating system that reader software works on? Or am I really supposed to go through the raw pdf source by hand?? Bleh...
Also, dont ask for my code - it's super dirty and unreadable. Instead look at the mupdf project. It supplies a renderer which is massively superior to poppler in terms of speed (even suitable for embedded devices) and comes with a program called pdfclean which does the same thing my program did so that I was able to get the number I needed.
I recently discovered the timestamp counter instruction which solved a problem where I had to accurately benchmark a very small piece of code while putting it in a loop made gcc optimize it away with -O3.
static __inline__ unsigned long long getticks(void)
unsigned a, d;
asm volatile("rdtsc" : "=a" (a), "=d" (d));
return ((unsigned long long)a) | (((unsigned long long)d) << 32);
More code for other architectures as well can be found here.
When using that piece one has to take care that the code stays on the same processor, the processor doesnt change its clock speed and the system is not hibernated/suspended inbetween.
Using terminal applications instead of GUI applications has the definitive speed advantage of not having to spend some time on moving a cursor to a 2D coordinate on the screen but instead just doing a 2mm downward motion with one or two fingers. I wonder if something other than terminal applications will allow me to interact with my computer faster until the invention of direct neural interfaces.
I really like terminal applications - not only because of the speed advantage but also because they offer much more real-estate in terms of usable screen space as they dont waste space on stuff like buttons, menu bars or all those pixels wasted on separators and the space between UI elements. Starting to use the pentadactyl firefox extension was not only a huge browsing speed improvement for me but I could also finally use my whole frickin 1920x1080 pixels for viewing the website (well except for a 19 pixels status bar at the bottom).
scriptreplay(1), being part of the bsdutils package in
debian/ubuntu and the util-linux package in rpm based distribution will
probably also already be installed on your system and is one of those really
handy tools you did not know of before even though they were always there.
script is a program that you can use to capture a terminal session or console
application output whereas
scriptreplay is able to replay that session by
using a timing file that
script is able to output on stderr. Without the
timings file, the typescript will include all terminal interaction and is
readily readable with a text editor or printable or easily uploadable to a
pastebin (no more selecting parts of your terminal window with your mouse and
copy-pasting that into the browser). Without the timing file it is useful to
document a process - for example if you want to show others how a bug happened
on your system or for a homework submission you have to hand in.
A very powerful feature is the mentioned timingfile you can capture by
stderr output of
script into a file. With the use of
scriptreplay you can then watch your terminal interaction in real time.
While it would be kinda tedious to share your typescript and timingfile over a
pastebin so that a party would have to download those manually and use
scriptreplay to watch them, I imagined a kind of youtube for terminal
sessions. This would also solve the problem of all those youtube videos that
are screen captures of terminal windows - needlessly encoding text as moving
images and by that not only destroying the ability to copy&paste things but
also needlessly increasing the filesize.
I remembered having seen such a website years ago but dont manage to find it again. To show a proof of concept I prepared the following website:
I will probably never have the time to make a real webservice out of it but maybe something of this is useful for others.