Because development won't keep quiet

My Role With Spectrum So Far

So it’s been a few months since I buried Kraken in lieu of moving towards Spectrum.  I’ve been very happy with it from a server administrator and user perspective, but I haven’t done as much as I’d hope coding wise.  A lot of things I was going to fix, Jan fixes in no time flat. […]

The Future Of Kraken

Hello everyone. It’s been quite a few years at this point since I first started Kraken. At that time I was developing PyAIMt and PyICQt and a Jive Software employee approached me about writing a plugin specifically for Openfire (Wildfire at the time). At first I balked at it, not wanting to develop something that was locked into a single server. However, I looked at the availability of libraries out there under Java and also the opportunity to really dive into a project in Java, a language I had mostly ignored in the past, and decided that sounded interesting to me. Plus it gave me an opportunity to explore some alternative methods of handling things I hated about PyAIMt and PyICQt. So I bid farewell to the Py transports, put them out there for someone else to take over, and dove into the IM Gateway plugin. Years later, after Openfire no longer appeared to be a priority for Jive, I decided to pull away the plugin and develop it separately, as Kraken.

My original goal was to set up Kraken to communicate with any server out there and yet still maintain the same functionality I could embrace as a plugin for Openfire. I never got to work on that. Over time the libraries I depended on lost their developers and languished under lack of development, leaving me to have to fix my own issues with the libraries. This was ok for a while but to be frank, it has gotten old. On top of that I rarely get any form of useful patches or help, so I find myself not enjoying working on Kraken at all anymore. I’m no longer interested in being a one man show.

Over the years, I’d had a number of folk ask me why I don’t develop a Kraken like project based on libpurple. Well the answer was always — that’s not what Kraken is, but why don’t you consider firing one up! I saw many of those attempts come and go. However then I heard about Spectrum. While I was pretty stoked to see it being developed I wasn’t expecting much. But a couple of weeks ago I checked in on the status of Spectrum and it’s still going very strong.

After giving it some thought I talked to the lead developer of Spectrum about the possibility of joining forces on Spectrum. With it’s use of libpurple, I shouldn’t have to spend much time with the individual protocol support and can focus on actual functionality of Spectrum itself. It’s already got a lot of the functionality I wanted Kraken to have, including file transfer and integration with ‘all servers’. It lacks patches/plugins for Tigase and Openfire but I intend to fix that. All in all it’s where I want to be — working together with someone else and able to tap into the excellent work of the Pidgin folk.

So.. what’s going to become of Kraken? Well, the whole thing feels like breaking up with someone to me. It’s not easy to part ways with Kraken. That said it’s the right thing to do for my sanity. After talking with the folk over at igniterealtime.org, we decided we move Kraken back to igniterealtime.org, where it can be with it’s target audience and where it grew up so to speak. While there is no developer lined up to ‘take over’ for me so to speak, the code will be right there in case the Openfire devs need to update it for compatibility, or want to put some serious work into it. I’m going to attempt to restore my JIRA instance enough to pull the outstanding issues out of there and put them into the tracker at igniterealtime.org and the source base will be moved to their SVN tree. I’ll shut down kraken.blathersource.org and probably have it redirect over to igniterealtime.org. I’m sure we’ll get a forum or something set up over there for people to chat amongst themselves. All in all it should make for a pretty smooth handoff and I’m a lot happier knowing it’ll have a real home instead of just floating about in the sourceforge project graveyard.

It’ll probably be a few days or weeks before this all falls into place. I’ll post again when things are finalized with some links to where things are now. This blog will remain and who knows, maybe I’ll get my butt in gear posting to it more often again. lol =)

Thanks for everyone’s interest in Kraken over the years!

What Is BlatherSource These Days?

Well it’s clear I got out of the interest in blogging about my coding experiences. That said, if I ever DO decide to blog about that, this is where it will go. To be frank Twitter has become my primary place for yammering about programming related things, or whatever else may be going on in my life.

So what then is the purpose of this site? Basically it boils down to this is the “core site” for all of the projects I host at blathersource.org. That is primarily Kraken as I have moved most other things off to SourceForge. I don’t intend to take this part of the site down – - because I mean – - – why. =) It’s not hurting anything. But I’m not likely to update it very often at all. =)

that said, I wanted to “leave a note” here for any visitors I might have.

I know what you’re thinking … I’m inside you maaaaan!

Ok maybe not. I’ve been quiet for a bit as I’ve adjusted to my new job and various other projects. Plus… my father passed away a little over a week ago… While I miss him, I am glad he is no longer struggling with his failing body. Anyway, the point of me bringing this up is, I’ve been silent because I’ve been involved in a number of things.

I’m in charge of a number of projects nowadays. I figured I’d take a moment to go through some of those, throw out some kudos, and let folk know how things are coming along.

Kraken
Kraken, the new life of the Openfire IM Gateway plugin, has been coming along. I’ve added Facebook, MySpace, and SameTime support to it, though those are still works in progress. (with thanks to the SIP Communicator folk, Pidgin, and the GSoC folk who worked on Facebook support for SIP C.) One of my “primary-ish” goals coming up soon is that I want to make it work with more than just Openfire. I’ve been working up the details of that in my head. Might be a bigger undertaking … not sure … but I think it’ll be worth the effort. Kraken’s first actual release was a little while ago to fix some budding MSN issues.

jQQl
There’s a new QQ java API, jQQl, that Kraken is using that is primarily a port of LumaQQ with english docs, stripping out the client part, and fixes to make it functional again. Kraken uses it for QQ support. Many thanks to the LumaQQ folk for the project, even if it does appear to have been abandoned! =)

jMySpaceIMl
There’s also a new MySpaceIM library for Java. It’s not particularly complete yet but it’s coming along. Many thanks to Version 2 Software for getting this started, along with Srinivasan, for getting this started before I took it over.

OpenYMSG
OpenYMSG was moved to hosted on SourceForge and is thriving there as a couple of talented developers have joined up. It had it’s first release the other day.

JML
JML is an MSN messenger java API, which has been around a while and also which I took over a long time ago. It’s also been thriving under the addition of a new developer!

jOTR
I actually created a project that was intended to be a java OTR library API, jOTR. The problem is, I no longer work on an XMPP client and have no use for this, so absolutely no development has been done on it. If anyone is interested in picking up the reigns and running with it, please let me know and I’d be happy to hand it off to you or just add you as a developer if you’d prefer.

Anyway. As you can probably see I’ve been involved in a lot of stuff. I’ve been working a lot with Drupal lately and wrote a module for it as well. I actually had someone express interest in my oldest XMPP related child, JWGC. =D Course that hasn’t worked with recent Jabber servers in ages but still cool to see. Someday I hope to get the C programming bug and play with that a bit more.

Amusingly enough, I was all prepared to put up some projects for the XMPP foundation GSoC group this year and it turns out they’re not doing it. ;D No worries, just thought it was funny on my timing.

So anyway, that’s been Daniel’s life lately. I’ve also been playing a lot of Phantasy Star Online – Blue Burst on the schtserv.com servers. =D Just throwing that plug in there. lol

BlatherSource is now running under Drupal!

A Quick History Lesson

You may recall that not so long ago, I switched my blog site from Serendipity to Clearspace. At the time I was working for Jive and was interested in having a real reason to “know” Clearspace and write plugins for it and such. That said, Clearspace is a completely beast for something so simple. Plus it required being run under Tomcat which I had little or no reason to run beyond for Clearspace. After parting ways with Jive, and not being interested in throwing money at a hosted server anymore, I moved my blog back to Serendipity and migrated the few posts I had made since moving to Clearspace to Serendipity.

The Now

A little while ago I realized that there was one thing I kept doing over and over again with my sites that weren’t really “for something”. Like for example, my personal site, our family site, etc. I would always do one of two things:

  • Come up with a “cool design” that I liked and never get around to writing content
  • Write out some small content that had no real design to it

A number of my friends are big fans of Drupal, and I certainly am not living under a rock, I had seen and played with it before, but without a purpose Drupal just seemed like “meh, yet another cms, what’s the point”. So having some extra free time last month, I decided to do something real with it — whip up a family site (http://vorpalcloud.org/) that actually had some form of look and feel to it, but also had some content, and was trivial to update/add to in the future.

So in the process, I happened to see how easy it was to manage Google AdSense units within Drupal, and I was attempting to see if I could get enough from ads to have the site pay for it’s own hosting somewhere. (as it turns out, not so much so far, but hey, if I decide to punt the ads at some point, then that’s trivial)

I had a number of projects out there (JML and such) that had crap sites and so I whipped up some Drupal sites for them and was pleased how those came out. At this point I’m running something like 13 sites under Drupal, including this one. And I’ve written my first Drupal module: Daily Twitter. Overall I’m highly impressed with the number of plugins, the community around it, and the product itself. I was able to whip up this theme for BlatherSource in very little time. The only thing I’m a little dismayed about is email notifications seem to be oddly hard to set up or something. Maybe I just haven’t found the right modules. Like I’d love to be able to subscribe to all content changes on a site but I can’t see any way to handle that. Likewise I was excited about making use of the project stuff but that’s not ported to Drupal 6 yet. But overall, I’d say this thing is fun to play/work with and has made it so I could whip up a large number of functional sites in no time flat.

So kudos Drupal! You made me enjoy managing my websites again.

How did I migrate from Serendipity to Drupal?

Well first off, keep in mind that I was the only user on my Serendipity site. It was my blog and my blog alone. I also never used the “extended body” stuff. But what it boiled down to is I did the following steps:

  1. Created database for new Drupal site (on the same MySQL server as my Serendipity blog)
  2. Ran the Drupal installer against the new database, but don’t go to “View Site” at the end of the install.
  3. Logged into the database as root.
  4. “use”d the serendipity database
  5. Ran the following SQL, knowing that drupal_bs is my Drupal database:

    INSERT INTO
    drupal_bs.node(nid, vid, type, title, uid, status, created, changed, comment, promote, moderate, sticky, tnid, translate)
    SELECT id, id, 'blog', title, '1', '1', timestamp, last_modified, '2', '1', '0', '0', '0', '0' FROM serendipity_entries;

    INSERT INTO
    drupal_bs.node_revisions(nid, vid, uid, title, body, teaser, timestamp, format)
    SELECT id, id, '1', title, body, body, timestamp, '2' FROM serendipity_entries;

    INSERT INTO
    drupal_bs.comments(cid, pid, nid, uid, subject, comment, hostname, timestamp, format, name, mail, homepage, status)
    SELECT id, parent_id, entry_id, '0', title, body, ip, timestamp, '2', author, email, url, '0' FROM serendipity_comments;

    UPDATE drupal_bs.comments SET uid = '1' WHERE name = 'Daniel Henninger';

    INSERT INTO
    drupal_bs.node_comment_statistics
    SELECT nid,timestamp,name,0,count(cid) FROM (SELECT * FROM comments ORDER BY timestamp DESC) q GROUP BY nid;

    UPDATE drupal_bs.node_comment_statistics SET last_comment_uid = '1' WHERE last_comment_name = 'Daniel Henninger';

    INSERT INTO
    drupal_bs.vocabulary(vid,name)
    VALUES(1, 'Serendipity Category');

    INSERT INTO
    drupal_bs.term_data(tid,vid,name,description)
    SELECT categoryid,'1',category_name,category_description FROM serendipity_category;

    INSERT INTO
    drupal_bs.term_hierarchy(tid,parent)
    SELECT categoryid,parentid FROM serendipity_category;

    INSERT INTO
    drupal_bs.term_node(nid,vid,tid)
    SELECT entryid,entryid,categoryid FROM serendipity_entrycat;

    Note that I also changed all comment references to myself to my actual uid (1) in Drupal.

  6. Done, go to site and all data transferred over.

I wanted to share this in case it helps someone else though.

Glorious Tricks w/ SourceForge

Edit: This is not working out like it’s supposed to. Still getting a svn-lock error when trying to init the sync, so while this in concept was a good idea, it’s not “taking”.

Pre-warning: Do not do this unless you know what you are doing, it will wipe the entireness of your SVN history for your project. It will be as if you just created your project fresh.

I wanted to svnsync an old repository to a new one on SourceForge. In my haste, I started setting up trunk/branches/tags without thinking. Of course, if the repo is not at revision 0, then you don’t get to svnsync. So I pinged support to get them to reset my sourceforge repository. After a little bit, it turns out I effectively had the tools in my own hands the whole time!

I’m sure you well know that there’s no way to wipe an svn repository completely down to revision 0 remotely. (at least none that I’ve seen so far) Well SourceForge has a nifty little tool (in your project, go under Admin, and subversion) to migrate your repository from cvs to svn. Well it didn’t seem very clear on that particular page, but you can -also- migrate from an SVN dump to your new SourceForge repository from the same tool.

So how does this fit together? Basically what you do is you create a local svn repository that’s completely empty:

> svnadmin create /tmp/myrepo

Then you get a dump of it:

> svnadmin dump /tmp/myrepo > svndump
* Dumped revision 0.

Then you gzip it (not sure if this is absolutely necessary):

> gzip svndump

Then upload that dump to your SourceForge file repository uploads directory, I used sftp:

> sftp USERNAME@frs.sourceforge.net
sftp> cd uploads
Connecting to frs.sourceforge.net...
USERNAME@frs.sourceforge.net's password:
sftp> put svndump.gz
Uploading svndump.gz to /incoming/U/US/USERNAME/uploads/svndump.gz
svndump.gz 100% 185 0.2KB/s 00:00
sftp> quit

Then you go to the migration tool that’s mentioned under Admin -> Subversion — Look for “Migration Instructions” and click on the migrate link under it.

You’ll see the svndump.gz you just uploaded. Make sure to click the Replace checkbox and then click Submit. Wait for it to complete and you are done! Back to revision 0 for you!

I hope this helps others who run into the same issue!

Chat clients and transports: Why do I see this over and over?

I tend to see the same set of things done over and over again in clients as they go from “I only care about XMPP chatting” to “users keep asking us for better transport support”. Many clients look for bare JIDs in your roster and treat them in a special manner, which is fine except in some cases I have bare JIDs in my roster and the service it’s attached to is gone, so the client gets a little confused. Clients like Psi will dutifully look for jabber:iq:register and offer that as an option on services. Many of them you have to go disco browsing to find the transports available in the first place. Nothing wrong with any of that. Some clients may choose to go ahead and look for things marked jabber:iq:gateway and add them to some special list. That’s not really what this is about though.

A number of clients will add in basically “hacks” for individual chat services. Psi has “service icon sets”, and if it’s a new service that isn’t in Psi’s list, it comes up with default icons. Spark has a bizarre mechanism in which it explicitly adds support for each individual transport via what I call little java “nuggets”. But a common theme I see here is individually handling each service. Some new service comes along, now all of the clients have to be updated if they want to have cool icons. Personally I think that’s crap. ;) I’m a big fan of dynamic lists of things. In other words a client shouldn’t have to -know- about any of the specific transports. It should be able to simply pull a list of services on the server in question, if it wants gateway ones look for jabber:iq:gateway, and — and here’s the missing part — be able to retrieve whatever it needs to provide a nice experience to the end user.

So what all tends to be custom added? Generally it boils down to icons. Icons icons icons. Icons for presence, Icons for representing the transport, etc etc. I don’t think I’ve ever seen anything beyond icons that couldn’t be determined from disco queries. So why couldn’t service icons be provided by the service itself? I could see something along the lines of using pubsub where the transport publishes it’s icon sets, one for service icons (maybe a set of different standardized sizes) and one for presence representation icons. That way a client could just come in and go “gimme the icons I should use” and move on. I admit my knowledge of pubsub is -still- limited, but my understanding is that there are version numbers associated with the published data, and the client could cache what it got and simply check for “is there a newer set?”.

The primary caveat here that I can think of is, that pulls the ‘control’ of the icons away from the client and into the transport, and there’s always a chance that the client doesn’t “like” the icons the transport is sharing. Of course, for the most part the clients are aiming to use icons similar to those of the service the transport is providing, so they kind of already have to deal with the possibility that for example AIM’s icon might not look great in their client. So the question is … would client authors use this or just end up scoffing at icons coming from the transports themselves?

I suppose if the clients really felt like they needed to have different icons for some services, they could always override what they’re getting from the transport and do their own anyway. It just always feels like a hackjob to me to see things like “if this JID matches with aim.* then use the AIM set”. Especially those instances make me cringe because JIDs can be anything for those transports, where the transports -will- be providing a gateway identity that tells you flat out what service it is, assuming the transport authors bothered to follow spec. ;)

Thinking about Gateways

It’s been a little while since I thought much about my illustrious IM Gateway plugin for Openfire. My previous job kept me a tad too busy with other things for me to really have time for it, or be interested in spending my free time on it. Now, let me prefix this by saying I don’t have a lot of free time right now, and I may not for a while. That said, I had some time to step back and look at the IM Gateway plugin and think about some possibilities.

So first off, I don’t really know the state of the Py*t transports. Last I heard there was no one maintaining them really anymore. That is a shame as I was hoping to see them flourish outside my involvement. If that’s not true, then I certainly apologize — just know it’s because I haven’t kept up well. Regardless, I found that I prefer Java to Python for the most part anyway. So would I take those projects back if they were offered to me? Probably not to be frank.

But the IM Gateway plugin — The fact that it only works with Openfire I feel is a disservice to other server implementations, but overall it’s a very cool implementation that tackled some issues with external implementations. But if you need to restart the IM Gateway plugin, you have to take the server with it. Java seems to have a habit of being very plugin-able friendly but then easy to break things with it’s container. I’m seeing that with an application server I’m working with — it’s not hard for an app to break the application server.

So I began thinking about what core things the IM Gateway plugin does that the Pys did not. Primarily it revolves around direct internal access to user’s rosters so that things could trivially be kept in sync. Things being, nicknames, groups, actual rosters, etc. So pulling back I’m wondering how feasible it would be to do something like this:

1. Strip away the ties of the IM Gateway plugin to Openfire and make the implementation standalone
2. Take that stripped away part and turn it into a small “helper” plugin that works with the external gateway implementation to accomplish the same goals
3. Similarly, write a small “helper” plugin for ejabberd that does the same thing
4. What about other servers I don’t have interest/time to write helper plugins for? Well then the transports would act like any other external transport.

How would these helper plugins do their jobs? Well somehow they’d have to have open communication back and forth with the actual transports to manage rosters, maybe even intercept packets and munge them a bit. A little ugly but hey.

As you can probably tell, over the years I have never come up with some solution that could be formed into an XEP for all of this. At some level I continue to help that I’ll see the light through my experiments here, but I’m not holding my breath. In the meantime I’d like to see solid transports that work for everyone. Writing an ejabberd plugin would give me a real reason to learn erlang and really see what it can do.

Now, if you are someone who is excited about this concept, please understand that I don’t have a lot of time right now. I need to spend my free time on things that make me money for the time being and unless someone is dieing to sponsor this concept, I couldn’t put it ahead of other assorted things I am working on. That is not a beg for money, that is a statement of my situation and why I’m not diving into this “right now”. ;)

I would love to hear feedback on this btw.

IntelliJ IDEA and Eclipse

So you may recall I posted about some problems I had with IntellIJ IDEA previously ( http://blathersource.org/blog/archives/73-IntelliJ…-Can-we-talk-about-this.html ). Since that time I had managed to repair the bizarre problem with it by… well completely whacking my entire home directory configuration of it. I use a Mac so it was in Library/Application Support and Library/Caches and maybe even preferences. That said, after cleaning all of that out and restarting IDEA, the memory leak weirdness had stopped occurring and all was well. Of course I had to go reinstall all my plugins and such, but hey. So hopefully this info will help others if they ever end up running into this same problem. So uhm.. Google index this please. ;)

You also may recall that I posted about trying out Eclipse ( http://blathersource.org/blog/archives/74-Trying-Eclipse.html ). I got a lot of great comments about it, pointing me in the right direction. Plus my previous employers had a group of folk who were highly into Eclipse and had some good advice. Of course, once I got IDEA working again it was hard to argue with myself to spend time getting used to Eclipse whereas that time could be spent coding.

So flash forward to no longer working with Jive Software (as of around mid October for those who are not aware). I no longer have a license for IntelliJ IDEA, so in working with another set of projects, I found myself firing up Eclipse again and guess what … I adore it! I feel like it really lets me really customize it to how I want it to feel, I love the level of plugin support for various things in it, and free is great. ;D I find more resources with it though. If I run into a confusion point I usually can quickly google an answer. Plus it’s not hard to find a chatroom or something to pop into to talk to real people “live”. If I “let go” of what I’m used to with IDEA, I actually find it more intuitive. I don’t really feel like going into serious details of what I like about it compared to IDEA, but suffice to say I am highly pleased with it and likely will stick with it for future java programming. =) I actually even though of writing up a document on things I learned while converting. (like terms that IDEA uses and what they map to in Eclipse) Who knows if I’ll ever write that.

Why I left the XSF

I’ve had a couple of people ask me why I did not reapply for membership in the XSF, so I thought I might post a bit about it here. I joined the XSF about 2 years ago not really knowing what was involved in membership. No one seemed to be able to tell me anything that being a member did aside from voting on the council, or other members. In the time I spent on XSF, that is literally all I did. It seemed like all of the “important discussions” had better places to be discussed at, like the council list, the standards list, and even a few more. I think it boiled down to, what’s left to talk about or even do? I don’t have any interest in being a member of something just so I can say I’m a member of it, so I chose not to rejoin this year.

If there was something to being a member in the XSF that I was missing out on, please, speak up and help me understand. =) And I certainly don’t have any ill feelings towards the membership, so please don’t take this as a “bash”. ;)

Previous Posts