return to OCLUG Web Site
A Django site.
January 7, 2011

Michael P. Soulier
msoulier
But I Digress
» Fixing my EeePC

So, close to a year ago I bought a new netbook for myself, a carefully researched EeePC from ASUS, mainly for the reputation of quality and Linux compatibility.

Last night I foolishly left it on but unplugged it, so by the morning the battery was dead. I didn’t realize this until I turned it on, unplugged, and found that it wasn’t working. The battery indicator clearly showed that the battery was too depleted, but oddly the box didn’t even attempt to boot. I plugged it in and it didn’t help. Nothing on the screen, not even a bios screen.

Calling ASUS tech support was no help, they finally concluded that I needed to send it to them. Meanwhile I was doing research online and found this extremely helpful thread that suggested a corrupt CMOS and a simple short-circuit technique to clear it.

So, before sending it to ASUS I pulled out my multimeter probes and shorted the CMOS terminals as described. Lo and behold, it booted. I’ve powered it down and back up repeatedly to confirm, and it’s good now.

So, ASUS tech support, 0. Eeeuser community 1.

Thanks guys!


July 23, 2010

Michael P. Soulier
msoulier
But I Digress
» I really love *nix

So I’ve recently been playing with Ditz, a ruby-based distributed issue tracker, to go along with my distributed workflow in Git. It’s a good start, but not quite polished yet. I added the issue-claiming plugin, played with it for a while, and then realized that I don’t need it since I’m the only developer on the projects that I want to use it for.

Then I removed the plugin, but it left behind sections in the ditz yaml files that caused it to now spew warnings.

msoulier@egor:...ier/work/mbg-bugs$ ditz todo
warning: unknown field "claimer" in YAML for
tag:ditz.rubyforge.org,2008-03-06:issue; ignoring
warning: unknown field "claimer" in YAML for
tag:ditz.rubyforge.org,2008-03-06:issue; ignoring
warning: unknown field "claimer" in YAML for
tag:ditz.rubyforge.org,2008-03-06:issue; ignoring
warning: unknown field "claimer" in YAML for
tag:ditz.rubyforge.org,2008-03-06:issue; ignoring

Well that’s unacceptable. So now I need to remove this claimer line from each file. Well, this is *nix so I’m not doing it by hand. I could use a perl one-liner but I’m a tad more familiar with ex commands, editing in Vim all day as I do.

So, I make an exscript file containing this:

%g/^claimer:/d
wq

And then run it on the files like so

for file in $(find bugs -name "issue*.yaml")
do
   ex - $file < exscript
done

Presto. Fixed. So happy.


June 11, 2010

Michael P. Soulier
msoulier
But I Digress
» Twisted Python and Chunked Encoding

When I was first writing a little web service in Twisted Python that would return JSON encoded data, and I was having some issues with loading it up using Javascript, I used Wireshark to trace the whole thing and was surprised at how the response looked.

There were delimiters around the data, and the response headers included a reference to “Chunked Transfer-Encoding”. I had to look it up to find out what it was, and I had no idea how to turn it off so I posted on the Twisted Python mailing list, and got a prompt reply.

Chunked encoding has nothing to do with the content type. It is used if
you do not set a content-length header.

So, figure out your response’s length (in bytes), and set the
content-length header to that.

Aha! So this in my http.Request handler fixed it.


log.info("sending response")
# Set the content length so that we don't respond with chunked
# encoding.
size = len(content)
log.debug("content length is %d bytes" % size)
self.setHeader('Content-Length', size)
self.write(content)
self.finish()
log.info("done")

Well, not a fix really as there was no bug, but I wanted to rule out the chunked encoding as the source of a problem that I was seeing.


» Humour in manpages

I just discovered surfraw in the results of an apt-cache search (love that command) and I had to laugh at the manpage:

DESCRIPTION
       Surfraw provides a fast unix command line interface to a variety
       of popular WWW search engines and other artifacts
       of power.  It reclaims google, altavista, dejanews, freshmeat,
       research index, slashdot and many others  from  the
       false‐prophet,  pox‐infested  heathen  lands  of html‐forms, placing
       these wonders where they belong, deep in unix
       heartland, as god loving extensions to the shell.

I know, I’m a geek, but to me it’s funny.


June 9, 2010

Michael P. Soulier
msoulier
But I Digress
» Dual-screen hack with x2vnc

I have two LCDs at work from having both a Windows and Linux PC on my desk. I need both right now due to some inane dependence on Windows-based process tools that were not of my choosing, but lets not go there.

To keep myself as productive as possible, I really just want to shove the two LCDs together and use one keyboard and mouse with a dual-screen setup. With one being Linux and the other Windows I won’t be able to move windows between the two displays but that’s ok. In the future I could put both LCDs on a single OS and access the other remotely but right now this is sufficient.

To make this work I’m leaning on x2vnc for a lovely dual-screen hack. Unfortunately it didn’t work initially, as the latest x2vnc speaks VNC 3.3 and TightVNC speaks 3.8 with no option that I could find to be backwards compatible. So, I looked around and I found RealVNC. The free, GPL version does have a backwards compatability mode of (use protocol 3.3 only) which I enabled, and now it’s working.

The VNC window on my Linux box is a thin, single pixel border on the east side of my monitor, and when I cross it I end up on the Windows box due to Microsoft’s ludicrous idea of a multi-user platform. So, thanks to the x2vnc developers, the RealVNC developers, and hey thanks Microsoft for being so lazy. It works great.


June 5, 2010

Michael P. Soulier
msoulier
But I Digress
» Cross-Origin Requests in Twisted

I’ve just been learning about Cross-Origin Resource Sharing, to permit javascript downloaded from one domain to make Ajax requests out to another domain. I started learning this because I was writing a Google Maps client to test some back-end code and it wasn’t working for some reason. Thanks to the help of someone on the Prototype mailing list, and a packet trace, the problem was quickly found.

When I loaded my static page off of the disk, the browser assigns it an origin of null. I was then accessing a service running on my desktop, so its origin was localhost. As the origins differ, when I tried to make an Ajax request to it my browser automagickally makes an OPTIONS request to the server, requesting permission.

Let me show an example, captured via tcpdump:

sudo tcpdump -i lo -nn -s0 -w out.pcap tcp port 8000

When I load up this pcap file in wireshark and follow TCP stream, I see:

OPTIONS /route/?start=sta-9998&end=sta-9999&starttime=1274469161 HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.4) Gecko/20091206 Gentoo Firefox/3.5.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Origin: null
Access-Control-Request-Method: GET
Access-Control-Request-Headers: x-prototype-version,x-requested-with

This is the OPTIONS request to the server, asking if it is permitted for this client to make a cross-origin request to that server. Specifically, it is asking permission to make a GET request from an Origin of “null”. If the server doesn’t respond with the right access-control headers, the browser will not permit the GET request to take place.

I had to modify my server, written in Twisted Python, to respond with:

HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
Access-Control-Allow-Headers: x-prototype-version,x-requested-with
Content-Length: 0
Access-Control-Max-Age: 2520

So here I’m saying, yes, it is permitted from any origin (hence the *) to make a GET request, and the client can cache this permission for 2520 seconds (42 minutes). This won’t be my response when I deploy, I will tightly control the domains that this service permits, and lower the max-age to more like 10 minutes.

Now, this initial response is not enough, be aware. These headers must be supplied in every response, not just the response to the OPTIONS request. So when the GET finally takes place it looks like:

GET /route/?start=sta-9998&end=sta-9999&starttime=1274469161 HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.4) Gecko/20091206 Gentoo Firefox/3.5.4
Accept: application/json
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
X-Requested-With: XMLHttpRequest
X-Prototype-Version: 1.6.1
Origin: null

And the server now responds with:

HTTP/1.1 200 OK
Content-Length: 76
Access-Control-Allow-Headers: x-prototype-version,x-requested-with
Access-Control-Max-Age: 2520
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET
Content-Type: application/json

{
    "reason": "No workers ready, try again soon",
    "status": "defer"
}

This is just an example while the server is loading a rather large data set, and cannot respond yet. Note the Access-Control headers in the response, just like the initial OPTIONS response.

Doing this in Twisted is simple enough. Inside of a http.Request handler, you can set response headers with self.setHeader(header_name, header_value), like so:

            self.setHeader('Access-Control-Allow-Origin', '*')
            self.setHeader('Access-Control-Allow-Methods', 'GET')
            self.setHeader('Access-Control-Allow-Headers',
                           'x-prototype-version,x-requested-with')
            self.setHeader('Access-Control-Max-Age', 2520)
            self.setHeader('Content-type', 'application/json')

My next steps are to tighten this granting of access, probably via configuration file, but I’m sure you get the idea.


April 9, 2010

Michael P. Soulier
msoulier
But I Digress
» Hydro Ottawa (aka. Need a longer lasting UPS)

Lately living in Stittsville is feeling more and more like being in a third-world country, given the frequency of power outages lately.

Two days ago the power was out in the morning for over an hour, and I look at the outage report and it says this:

2010-04-07  	Goulbourn  	7:42 am  	9:17 am  	95 minutes  	3325
    • cause unknown, line was patrolled and no cause was found

And this morning it went down again. The website doesn’t show this one yet (they seem to update the site as an afterthought) and when I called into their irritating phone queues the human being that I finally reached had no information.

Reliable power? Not here.


December 12, 2009

Michael P. Soulier
msoulier
But I Digress
» Ruby silliness

Ok, this is just dumb.

msoulier@kanga:~$ gem list torrent --remote

*** REMOTE GEMS ***

Well that’s wrong, I know there’s a RubyTorrent gem.

msoulier@kanga:~$ gem list tftp --remote

*** REMOTE GEMS ***

tftpplus (0.4)

It finds my tftp library just fine with a substring.

msoulier@kanga:~$ gem list RubyTorrent --remote

*** REMOTE GEMS ***

rubytorrent (0.3)

So why do I have to be so specific?

I shouldn’t need a web interface to find code in a repository people! Learn from apt-cache.


November 20, 2009

Michael P. Soulier
msoulier
But I Digress
» Java time capsule

I’ve been involved in some discussions regarding Java recently, and I’ve repeated said that I mostly find it a solution that is still looking for a problem.

Looking back at this post by Paul Graham on “Java’s Cover” I find it interesting how many of his points still ring true, 8 years later.

My favorite quote:

It could be that in Java’s case I’m mistaken. It could be that a language promoted by one big company to undermine another, designed by a committee for a “mainstream” audience, hyped to the skies, and beloved of the DoD, happens nonetheless to be a clean, beautiful, powerful language that I would love programming in. It could be, but it seems very unlikely.

My problem with it is simple, and it’s why I dislike ClearCase, and many other technologies; it makes easy things hard. I’m busy. I’d use it if forced to, and then I’d try desperately to like it. Until then, I have better things to do.


November 18, 2009

Michael P. Soulier
msoulier
But I Digress
» Java sucks

Well yes, we know this already from the way that it makes easy things hard, and hard things nearly impossible, but it’s rarely been captured with the eloquence that I find in these wonderful quotes on the topic.

And yet, as cockroaches are to humans this technology/language/marketing campaign continues unabated until having java on one’s resume is a requirement to find a job through some ignorant HR department that has no idea what is is. Like the job posting I saw a few years ago for a java programmer with 10-years experience when java was only 7 years old. Luck with that.

My favourite quote:

If Java had true garbage collection, most programs would delete themselves upon execution. — Robert Sewell

My boss asks me occasionally why I don’t use Java, and I tell him that I have many tools in my toolbox, some good and some bad that I bought on impulse due to good marketing or simply because they were new. Java is like my trendy flip-grip pliers from Crappy Tire that try to be both clippers and needle-nose pliers, but suck at both jobs. I don’t hate the tool, but I certainly don’t reach for it often, and I’m thinking of throwing it out.


August 18, 2009

Michael P. Soulier
msoulier
But I Digress
» Call Select does the right thing

I just got a few calls, two of them from Call Select. The first one asked me to call back and the second said that the matter was cleared-up and we would not have to pay anything, meaning that they can’t find any proof that we ever agreed to their service in the first place.

Why they didn’t just immediately absorb the buck-and-a-half of fees incurred is beyond me, was it really worth the hassle? The person I talked to claims that they most-likely made a mistake. Possible, so I shouldn’t overreact here. The fact is that I don’t know, regardless of what my cynical side wants to suspect, so I’ll just let it lie. I’m more interested now in why this kind of thing is possible in Canada.

Anyway, I also got a call from a reporter a the Ottawa Citizen, as I did email him about the issue, and he said he’d been looking into it. I’m not a big believer in coincidences, personally, so I think I’ll go buy the Citizen for at least the next few days. I suddenly like them even more than usual.

If the Citizen wants to follow-up on weak laws around telecom in Canada, more power to them. Maybe it will help the next guy.


August 10, 2009

Michael P. Soulier
msoulier
But I Digress
» Mike Z., you’ll be missed


…not. Thanks for nothing. I guess six-sigma doesn’t really fix everything, huh?

August 2, 2009

Michael P. Soulier
msoulier
But I Digress
» Still No Long-Distance


So, thanks to our long-distance service being taken over by Call Select without our permission, every time I try to make a long-distance call now, I am asked for an access code.

I was told by a Bell rep that it would be fixed by now, but it’s not and they’re off until Tuesday (long weekend), so no long-distance calls for us.

Thanks again Call Select. And thank you CRTC for working so hard to prevent this kind of thing.

August 1, 2009

Michael P. Soulier
msoulier
But I Digress
» Follow-up to the CRTC


I followed-up to the CRTC on my complaint with this, just out of my own curiosity.

Thank you for responding. The issue is resolved for now, my only concern is how easy it was to “slam” my line in the first place.

How is it that anyone considers the current system to be in any way secure, where some random phone company can claim that I requested their service without any authentication from me whatsoever? Is that not an obvious flaw in the system that should be addressed at some point?

If the CRTC is not regulating the rates, terms of service or business practices of long-distance service providers, then who is?

Thank you.

» Response from the CRTC


So the CRTC responded.

From: CRTC DONOTRESPOND/NEPASREPONDRE
To: msoulier@digitaltorque.ca
Subject: Your message of 7/29/2009 6:20:15 PM (Reference Number: *****)

Dear Mr. Soulier,

Thank you for taking the time to contact the CRTC about your concerns.

The CRTC no longer regulates the rates, terms of service or business practices of long-distance service providers.

If your long-distance telephone company is switched to another company without your permission, it’s called “slamming.” Slamming is not condoned by either the industry or the CRTC. To find out who is providing you with long-distance service, call 1-700-555-4141 from your telephone and check your monthly telephone bill. Then, report the transfer to your original long-distance company immediately and ask to be switched back. Pay only the rate you would have paid to your original phone company and file a complaint with the company that switched your service.

If you’re not satisfied with their response, check the Commissioner for Complaints for Telecommunications Services (CCTS) to see if your service provider is a member. If so, you can contact the CCTS with your complaint.

The CCTS is an independent agency that helps resolve your complaints about your telecommunications service. Contact them at:

* email: info@ccts-cprst.ca
* mail: P.O. Box 81088, Ottawa, Ontario K1P 1B1
* fax: 1-877-782-2924
* toll-free telephone: 1-888-221-1687
* toll-free TTY: 711 or 1-800-855-0511 (voice)

I am also providing you with a link to the Fact Sheet entitled “How to File a Complaint About Your Telephone Service” which explains the CRTC complaints process : http://www.crtc.gc.ca/eng/INFO_SHT/T12.htm

IMPORTANT NOTE: Please do not reply to this message using the email address indicated above as we cannot receive e-mail at this address. To reply or to add to your submission, please click here and follow the prompts: http://www.crtc.gc.ca/rapidsccm/landing.asp?lang=E&caseid=*********

Yours sincerely,

Chantal Proulx
CRTC Client Services

So it doesn’t look like anything will come of this to improve the situation. I’ll follow up with a few questions, but I don’t expect my one complaint to change much.

April 13, 2009

Michael P. Soulier
msoulier
But I Digress
» Big Gentoo upgrade today

Checking emerge for available updates for my Gentoo workstation, I was surprised to see a big jump in many packages. The reason was Gentoo pushing Xorg 1.5 as the new stable version as opposed to 1.3.

I know it works, as I’m already running it on my laptop in Ubuntu 8.10, but there it’s configured to use evdev and HAL, and I have HAL disabled right now in Gentoo to try to keep things light and fast, so I wasn’t sure what I’d run into, even after reading the upgrade guide. Nice that a –pretend emerge run pointed me at the news. I like prompts like that, they’re very helpful to me.

It took hours on my little AMD Athlon, and when I restarted X I hit this problem, which is technically my fault for not reading the notes at the end of the upgrade and rebuilding my mouse support. Thankfully someone posted to the gentoo mailing list about it, and I captured his note before restarting X, so all is well now.

An update in QT seems to have broken qbittorrent, and the next version isn’t stable yet, so I’ve switched to rtorrent for now. Some change is good as long as my whole box isn’t useless to me for long periods of time.

I’m still debating dumping Gentoo as too much work, but it’s forcing me to keep up-to-date with some changes in the community, so it’s not really a bad thing. We’ll see. Running bleeding edge is hard with older hardware that suddenly finds itself unsupported. Which is funny since older hardware is one of the best reasons to run Gentoo and keep the builds lighter than the prebuilt binaries from most distros.

Maybe I should bite the bullet and just build with HAL support. Obviously HAL isn’t going away.

April 9, 2009

Michael P. Soulier
msoulier
But I Digress
» Tftpy state machine overhaul

I just posted this news item to SourceForge.

I’ve decided that the state handling in tftpy is too difficult to maintain, and I’m ditching it. I’ve started that work in a private branch in Git.

First though, I’m going to merge all of the contributed patches into an experimental branch and push that to github. I’ll then rebase my state-machine branch on that and keep going.

It’s a big rewrite, so expect breakage in the short term. Contributed unit tests are welcome, I really need to flesh those out.

Cheers.

A merging I will go, a merging I will go…

April 7, 2009

Michael P. Soulier
msoulier
But I Digress
» Querying the db schema from SQLite

I’ve been trying to put together a migration strategy for SQLite that is not simply a bunch of versioned SQL fragments that are extremely difficult to backport.

The information is there but it’s not obvious. You can fetch the list of tables in an SQLite database with this little snippet.

SELECT name
    FROM sqlite_master
    WHERE type='table'
    AND NOT name='sqlite_sequence'
    ORDER BY name;

From there, you can simply loop on the table names and pull out all of the table columns and their types via

PRAGMA table_info($table_name)

I have some rudimentary code now in Perl that queries this and builds-up a multi-level hash of all of the tables, their columns and metadata. Pass that to each migration fragment and we don’t need a schema version anymore. Each migration fragment has enough information to conditionally do migration.

At least, I think it’ll work.

April 2, 2009

Michael P. Soulier
msoulier
But I Digress
» Building rpms out of Git

Having gotten heavily into using Git lately for a lot of my work, when I’m prototyping something and I want to make an rpm out of it, I find this script, kept at the root of my working files, quite helpful.

msoulier@kostya:...itel-msl-webproxy$ cat git2rpm
#!/bin/sh
	
specfile=mitel-msl-webproxy
	
version=$(grep Version ${specfile}.spec \
                 | head -1 | awk -F: '{print $2}' \
                 | cut -b2-)
	
fullname=${specfile}-${version}
	
git clone . /var/tmp/$fullname || exit 1
rm -rf /var/tmp/$fullname/.git*
	
dest=$HOME/rpms/SOURCES/${fullname}.tar.gz
rm -f $dest
tar -C /var/tmp -zcvf $dest $fullname && rm -rf /var/tmp/$fullname
	
rpmbuild -ta $dest

Here I’m building an rpm called mitel-msl-webproxy. All I do is clone the local repository to /var/tmp/$fullname, build a tarball out of that into my $HOME/rpms/SOURCES directory, and then run rpmbuild -ta on it, which works on tarballs that have specfiles at their root.

Dead simple. It should go without saying that even building rpms out of Git is a lot faster than building them out of ClearCase.

March 26, 2009

Michael P. Soulier
msoulier
But I Digress
» Why browser certificate warnings fail

Everyone’s probably seen one. You visit some website with a URL prefixed with “https” and you get a pop-up or warning of some kind in your browser, telling you that the certificate for the site is not signed by a known authority, and warning you not to continue. You continue anyway since, surprise surprise, you needed to go to that website for a reason.

Lately in more recent versions of Internet Exploder and Firefox, these warnings have become more obtrusive, and it’s on purpose. Browser vendors want you to have to work to get to a secure site with an invalid certificate, and it’s for more than one reason, good and bad.

  1. Websites running certificates not signed by a known authority can be put up by anyone, and the current site may not deserve your trust.
  2. DNS hijacking could direct your browser to a completely different website than you think you are visiting. The point of the host certificate is to ensure that you are talking to the people you think you are talking to.
  3. Valid certificates are big business, employing many people at Verisign, Thawte, etc. If just anyone can put up an SSL-enabled website then it undermines their business model.

I could care less about Verisign’s business model, I think that valid Certs are way too expensive so I run a self-signed one myself. Furthermore, I work on applications and infrastructure for a Linux distribution that has an SSL-enabled web interface for management. We want SSL to secure the user’s session key, and any privileged information being transmitted between the client and the server. But, we cannot afford to buy a valid certificate for each and every box. No way.

So, we compromise. We generate a self-signed cert and we provide a mechanism to install your own if you choose to buy one. Problem solved, right? Wrong.

We have teams here that don’t want customers to be scared off by the certificate warning when they first visit the interface. So, they just use unencrypted, insecure HTTP instead.

Yes, that’s right. They’re more afraid of the warning in the browser than the fact that the session is unencrypted, potentially over the Internet. So, what are the browser vendors accomplishing by making the warning more prominent? They’re encouraging application developers to stop using SSL.

Bravo.