2009-12-19

On Question/Answer social applications

I really like question answer social applications like StackOverflow.com because most of the times they are a better alternative to a forum. One of the creators Joel Spolsky explains the success at Learning from StackOverflow.com video.
The ideas that make StackOverflow.com so great are summarized at:
  • Voting: There needs to be a way to rank answers to a question up or down so you can see the best answer the first.
  • Tags: Hierarchical categorizing fails where tagging works wonderfully.
  • Editing: You need to be able to edit both questions and answers so they do not become obsolete.
  • Badges: Users should gain status by completing actions on the site that gain knowledge on the site features and make people use them.
  • Karma: Users should gain status by asking good questions and replying with good answers.
  • Pre-search: Asking a question needs a search first. It really prevents duplicates.
  • Google is UI: Google should be able to index the whole site for users to access questions.
  • Performance: A performant site is used.
  • Critical Mass: Users will only use sites that trust on. And stackoverflow.com was developed by two very influential IT bloggers.

2009-07-12

Web Applications should always use UTF-8 charset

I have seen that iso-8859-1 encoding in HTTP is broken as some "genius" from Microsoft decided to encode non iso-8859-1 characters using HTML entity encoding without using proper escaping, thus making non iso-8859-1 characters unusable and iso-8859-1 encoding for web applications not useful. FORM submission and i18n

Being Windows-1252 a superset of iso-8859-1 has made every browser and most other programs interpret Windows-1252 when iso-8859-1 is specified. Even the HTML5 standard states that iso-8859-1 should be interpreted as Windows-1252.

Apache Tomcat needs to use UTF-8 instead of iso-8859-1 by default. Here are some tips on working around this bad Tomcat default: Tomcat/UTF-8.

Always use UTF-8 as the encoding of any HTML page and especially of Web Applications.

2009-04-18

On Web bookmarking

The use cases I need for a web bookmarking application are:
  • Quick button or key combination to bookmark an URL.
  • Review bookmarks for obsolescence. Each bookmark should be checked periodically to find if the resource still exists and have a notification when it does not exist. Bookmarks should have attached a review date to be able to review bookmarks one by one.
  • Retitle bookmarks. (Many times the title on the page is not the one I want)
  • Classification. I do not like hierarchical classification because many bookmarks may own to more than one category and hierarchical classification can only have one folder structure. (Example: low fat meals/fish or fish/low fat meals) (tags also have problems as it is common to tag an AJAX tutorial as "AJAX,JavaScript" but AJAX already contains JavaScript implicitly on it but I want to get that page when I look for the broader term like JavaScript resources)
  • As bookmarks refer to online resources they may only be accessed online, but I still need transparent backups in case the online resource that hosts my bookmarks accidentally removes them or just disappears.
  • Pending revision. I sometimes see something that I need to review but I still do not want to publish in my public bookmarks.
I am currently using delicious.com to host my public bookmarks, but that solution alone lacks reviewing and quick bookmark saving (unless you use the fat Firefox plugin).

2009-03-17

On filesystem usage patterns

The ext4 data loss bug leaved some changed files with zero size in the case of a crash, making the API not transactional. The bug was fixed, not in the side of the application, but in the ext4 code itself with fixes to other delayed allocation filesystems like XFS and BTRFS to maintain semantics compatibility. I started to think about use cases both in servers and laptops to investigate if the current POSIX API of Linux and if the internal Linux filesystems implementations cover these use cases as best as possible.
  • Copying, moving, creating or replacing a huge file while saving a minor setting or browsing the Internet should not ever block your GUI application noticeably.
  • Modifying a file should always leave either the old one or the new one. (but never a zero length file!)
  • Small file change (substitution) atomic. Transaction complete should be notified but can be delayed for some amount of time like when in laptop mode.
  • Writes on the hard drive should always transition stable states to prevent any loss of prior data. As much only recent changes should be lost but old data should always be preserved.
  • Streaming
  • RDBMS tables
  • Laptop-mode: the system should be able to delay writes to disk to save power by only spining the hard drive each few seconds.

2009-03-11

On Self-extracting archives

Microsoft is still favoring self-extracting archives over normal archives. They probably want to sustain the antivirus software market by providing files capable of containing computer viruses and not having a sandbox to run these files by default in their badly designed "operating" systems.

Microsoft goes over to extremes like having a self-extracting EXE file that contains a CAB file that contains a ZIP file that contains the actual files (see for example Microsoft KB 303436 where they distribute such a file) (maybe they are so clueless that they think developers do not know what a ZIP file is and how to unpack it)

One way to prevent Microsoft incompetence (or maybe corruption) is to have in proper operating system never execute EXE files outside a sandbox and by default detect self-extracting archives (by file contents inspection instead of just only the file extension) and unpack them directly by the installed unarchiving software.

2009-03-01

Why compatibility with Microsoft may lead to legal problems

Microsoft sued TomTom (Microsoft sues TomTom over Linux and other patent claims) for patents like 5,579,517 and 5,758,352: "Common name space for long and short filenames". When TomTom was just using the badly designed FAT from Microsoft for compatibility with Microsoft products. TomTom (like other flash storage users) could just have used a decent flash filesystem like YAFFS. This is just another example why software patents lead to fragmenting compatibility between systems.

2009-02-28

On Integrated development environments

The main problems I find with the Eclipse IDE are:
The features I need from an IDE are:
  • Code refactoring: I want to be able to rename a method or variable without renaming other different equally named variables.
  • I need to know the callers of a function to know the consequences of changing its behavior.
  • I need VCS integration to know the current changes and be able to quickly change branches to develop new features.
  • I need automatic compilation and verification.

2009-02-08

Duplicate action detection and correct response

To prevent any action from being duplicated due to communication errors at the first try it is needed to identify the first communication with an id that the server has to verify to not duplicate the action.

The common case is like this:

A client sends a communication to a server requesting a new order. The communication has an orderid. The server gets all the info and performs the action. But that communication fails to get a response because of a network problem like loss of connectivity. Since the client is not sure the communication was totally OK the client must retry back the communication. Since the communication has an orderid there is no problem in sending it again (no risk of duplication). Since the server already performed the action, this second time the response code should be either just the same as the first case "OK - accepted" (for simplicity) or "OK - already accepted". It is critical this response be understood by the client as OK, otherwise the client will retry forever.

I am explaining this because I have had problems trying to make understand people about this problems that made some orders, although correctly sent to the server, never marked as OK on the client system and thus retried forever making the user unhappy since his order was like not being sent.

2009-01-06

Social Bookmarks Friend Finder (for delicious.com)

I am quite interested in social networks as they allow you to get in contact with people with similar interests.

I found some time ago a social bookmarking site called delicious.com that allows you to have all you public bookmarks tagged. I liked to find similar users to me but the site had no option for that and I found Find Similar Users on del.icio.us but it had some problems like excessive memory usage, no continue and the need to provide user and password to access the data. So I developed a similar Java program without those problems called Social Bookmarks Friend Finder (for delicious.com). It is coded in Java, using the embedded H2 database by default (being able to use most compatible JDBC databases). Please try it if you want and report any issues. I would like to hear if it was useful to you!

2009-01-01

URLs should never have the server side technology

URLs should never have the server side technology used in them. Because it is just marketing for that technology and it does not just identify a resource.

Technologies that are shown in the URL but should not (with URL examples):
  • CGI https://bugzilla.mozilla.org/show_bug.cgi?id=249338 my suggestion https://bugzilla.mozilla.org/bug?id=249338 or https://bugzilla.mozilla.org/bug/249338
  • PHP http://www.facebook.com/home.php my suggestion http://www.facebook.com/
  • JSP https://issues.apache.org/jira/secure/Dashboard.jspa my suggestion https://issues.apache.org/jira/
  • ASP http://www.microsoft.com/en/us/default.aspx You should not ever have to refer to a default.aspx as http://www.microsoft.com/en/us/ should always be enough (ASP.NET also has problems rooting in the ancestor of the current MSWindows CP/M)
Most of the times these are examples of the lack of correct MVC web architectures. Always try to not use technologies that force you to have the technology name in the URL. It is bad practice. And a clear example of bad architecting.

You should also prevent using the technology name in the URL as in:
  • http://help.open.collab.net/servlets/tracking
So, always try to call a controller that will render a resource (JSON, HTML, XML, or any other format)

On Programming Languages

Business code should never be developed using a non memory managed programming language. Being either reference counting or garbage collection (although reference counting is time predictive it can make your application more laggy and can not alone detect object cycles so garbage collection is the winner nowadays as Jamie Zawinski says). When writing business code you should not lose time managing memory when the computer can do it better for you.

The programming languages I recommend are:
  • Java (for most applications including business or enterprise applications)
  • Python (for any small application for computer administration)
  • C/C++ (only for low level or computing intensive code)
Some other programming languages that I neither recommend nor discourage:
I do not like scripting languages because I want my code to work flawlessly and not compiled programming languages do not allow me to catch any error at compile time.

2008-12-11

On Graphical User Interfaces

Apple Inc. has always been the leader in graphical user interfaces. Apple have always know very well how to create intuitive user interfaces. Apple has always know the difference between a document and an application and has centered the user interface around the document while Microsoft has done the same around the unintuitive and computer specific software application.

Users should not need to know about applications (they do not need to know they have to open the spreadsheet application to create a spreadsheet document) they just need to know about documents or objects and actions (they just need to know that they can click somewhere to create a new document and just clicking on a document it is opened for viewing, editing or printing). Applications should be hidden totally from the user.

I really hate applications that use multiple document interface mostly because they force the user to understand what an application is, instead of just focusing on the object (text document, spreadsheet, image, video, song, ...). It seems Microsoft Office 2007 has finally done some progress to stay away of MDI but Excel still has an internal close button (x).

2008-11-27

On Google Technologies

Google is currently the information technology leader because most of their systems are better engineered than the competence (like Microsoft or Yahoo). Here is a list of technologies that Google uses that you should be using too.
Python at Google (Greg Stein - SDForum) is a blog post about technologies used at Google.

2008-11-18

On SQL Databases

Choosing wisely a SQL database is a very important step for any software project that needs permanent storage and efficiently querying that data. So, here is my view of the current SQL databases.
Wikipedia's Comparison of relational database management systems
MySQL vs PostgreSQL - WikiVS

2008-09-27

Why you should buy Linux compatible hardware

I have always recommended buying Linux compatible hardware.

I have also seen along the years that most Linux supported hardware has more quality than hardware not supported by Linux because most Linux users choose wisely what is the best hardware to buy and they only develop drivers for them.

Always take a look at the level of support of any hardware prior buying it:
Is my hardware Linux-compatible? Find out here is a quite good article on hardware compatibility on Linux although a bit outdated.

2008-09-18

My ideal mobile device

Hardware requirements:
Software requirements:
Conclusion:
phonedog does some decent mobile phones video reviews.

Smartphone OS comparison

2008-09-11

Microsoft Internet Explorer Bugs

Most clients still have Microsoft Internet Explorer 6 as required web platform and I have to support it for web applications. Being Microsoft Internet Explorer such a badly designed browser have lots problems that make developers do workarounds to make the code run and prevent any crashes on MSIE. Microsoft will support MSIE6 until 2014 so the future of web innovation is endangered by Microsoft.

This is a (growing) list of bugs of Microsoft Internet Explorer make my life harder:
Web Bug Track is a great list of mostly MSIE bugs.

Please, STOP using Microsoft Internet Explorer NOW! or at least Bring Down MSIE6! Google is already taking steps by unsupporting MSIE6 soon. You may want to use IE6 Update to make your users upgrade.

Can Microsoft get any lower?

2008-09-08

C Libraries

These are some of the C libraries that I use and I recommend:

2008-08-31

On Mozilla Firefox

I do not understand why Mozilla Firefox developers are so slow to fixing some usability bugs like:
I am trying to fix some critical bugs related to saved files corruption like Bug 226970 – lost of all stored passwords if out of disk space file truncated to zero length but it is a tough endeavor since the OS abstraction library of Mozilla NSPR seems to be unmaintained as nobody replied to my email offering my help to fix these bugs.

Mozilla has also do some changes like unverified SSL-Certificate management UI that may not be in the users interests but in the CA industry ones. That UI has been thought to make users life much difficult when using non CA signed SSL-Certificates thus having websites pay for CA certificates. They could have just issued a big warning every time the user browses a non CA signed website instead of the ugly UI that may have been paid by the CA industry.

Mozilla has historically not collaborated with the libre opensource community, making Mozilla code huge, thus needing many developers to properly maintaining it. Until Firefox3 it had an in house graphics renderer but switched to the powerful cairo. Same with data storage where it used the braindamaged Mork file format (as Jamie Zawinski rightfully calls it) but switched to the powerful SQLite.

If Mozilla wants to succeed in the long run, they must collaborate with the libre opensource community by depending on other projects (and supporting them) instead of developing all their software in-house.

Mozilla Firefox now also has to compete with Google Chrome as it seems like Google, that pays up to 80% of Mozilla Firefox development, will cut the funding when Google Chrome surpasses in users to Firefox. Google would not have started Google Chrome development if they thought Mozilla developers would be good enough to deliver a quality browser and Google will not backup their plans since Google Chrome is already out of beta and bundled with other Google tools.

With all these problems Mozilla Firefox future seems to be quite unclear. I think Firefox will start losing all the users when Google Chrome has all the features that Firefox users currently have and a Linux and Mac OS X port is released.

This is the list of add-ons I consider a must have for Firefox:
  • Adblock Plus prevents advertisements
  • Tabberwocky opens new tabs just next to the current one.
  • FlashBlock only runs flash content when clicked on it
  • livehttpheaders see HTTP headers of all the requests. This is for developers mainly
  • Firebug is the best companion to help develop web pages. It lets you see the DOM, CSS, JavaScript, HTTP requests. This is also for developers mainly

2008-08-28

On Version Control Systems

Using a Version Control System is a must for any software development right from the start (even if your boss denies it). Here are some of my reviews:
My favorites are Git and Quilt and I consider CVS totally obsolete and Subversion is becoming obsolete for most development use cases but maybe to keep upstream versioning.

This is a list of features I need in a VCS (I got most from Version Control System Comparison):
  • See what is going to be commited (aka the full changeset) instead of just the modified, added, deleted files. With most VCSs I have to command diff and then command commit. This of course needs to allow aborting the commit operation. This feature is most useful to newbies that do not know that they always have to diff prior to commit to see if they see the changes they really want to commit or not.
  • Atomic commits (aka Changesets) are a must. CVS and the crappy Visual Sourcesafe do not have atomic commits. Most modern VCSs have atomic commits.
  • Files and Directories Moves or Renames. CVS does not have moves or renames but subversion and git do. Handling renames: svn vs. git vs. mercurial.
  • Remote repository replication. To be able to browse the history while offline. CVS and Subversion have some external programs to be able to do that. But you need a modern VCS like git to be able to get the history and work with it natively.
  • Propagating Changes to Parent Repositories and Disconnected operation. Be able to commit many changes without being online and be able to submit these changes upstream when I become online. CVS and subversion only support on-line operation so they are discarded.
  • Tracking Line-wise File History. I want to know who and when changed some line. cvs annotate, svn blame, git blame, ...
  • Networking support with HTTP and HTTPS to be able to access the repository behind a HTTP proxy. subversion and git can support it while CVS does not.
  • Easy branching, merging and rebasing: When I need to code a new feature or fix a new bug I want to base my new code in the latest stable code, not in the work-in-progress feature that it is not finished yet, and be able to work on many features in paralell without any interference between them. Whenever these features are ready and tested I want to easily merge them to mainline. This means being able to change branches quickly is a must. Since mainline could have been changing I want to be able to rebase my changes to the latest mainline. CVS, Subversion and perforce make merging painful so they are discarded. git is great for this kind of operation.
  • Easily revert a merged feature. I do not mind losing the history of the coding of the feature but it must be easy to revert them.
  • I want to easily make changes to upstream projects that are constantly changing and rebase my changes easily whenever I want.
  • Partial Pull: Be able to get just the last revision instead of the whole history of a project, just like CVS and subversion do. Monotone is not able to work just with the last revision and you have to get the huge whole history of the project.
  • I want to see merge requests from my developers and with a click accept the merge request or reject it with a comment. So developers should be able to submit merge requests. I do not want to lose any merge request and multiple merge requests of the same branch should be merged (so email will not work here).
  • Be able to completely remove a commit or file, committed by error or that may contain confidential information that should never be in the history. Subversion has problems with this use case (How to remove an accidentally put large file (4GB) from Subversion repository) but git lets you rebase the project history quite easily (Collapsing a git repository's history)
Martin Fowler has a good article on Version Control Tools

2008-08-07

On JavaScript WEB Application Frameworks

I have found jQuery to be the best library for JavaScript web development. jQuery has the biggest community, more high quality plug-ins than any other JavaScript library and I have found it is the best designed library. Most projects are moving to jQuery from other JavaScript libraries like Prototype JavaScript Framework (See this article of a developer that migrated from Prototype to JQuery and this ticket about the midgard migration from prototype to jQuery) or like Dojo or YUI (Ning moved from Dojo to jQuery Djblets and Review Board moving from YUI to jQuery anyMeta moves from Dojo to JQuery). I still have to check out Google Web Toolkit (it seems like the ideal bet, but it uses its own GWT-RPC protocol by default that is not a standard web service interface). (See this Evaluation of JavaScript Libraries and reddit comments on it). Now, that Microsoft and Nokia are supporting jQuery it is by no doubt the best option to choose. Why does everyone like jQuery more than prototype/script.aclo.us or mootools or whatever?

Comparison of JavaScript frameworks

These are my favorite jQuery plugins:
  • jQuery UI high quality basic widgets
  • Tablesorter makes any HTML table client sortable
  • History prevents the client application from breaking the back button
  • Form Deserialization deserialize JSON to a HTML form
  • Form makes any form AJAX ready with little code
  • Autocomplete text input autocompletion (both local and remote even using JSON)
Most browsers are getting or are quite optimized for JavaScript execution like:
See this comparison of JavaScript engines by John Resig of jQuery fame.

That means that Client Web Applications will run much faster and will allow for much more than is currently possible. So, it is time to start betting for a future of JavaScript Web Applications that contact the server using XMLHttpRequest and REST WEB Services. I also bet this will put dynamic server web pages technologies (like JSP, PHP, ...) on extinction (also because these technologies do not use DOM to manipulate XHTML and I think it is the only correct way).

A web indexer (like Google) just needs to access the raw resources using REST. It should not crawl any web application. And these resources should be raw (without any decoration like navigation menus) and just one resource per URL.

2008-08-04

My Boycotts

This is the list of my boycotts:
These previous companies create such alternative technologies just to lock-in users and have them pay more than they would in a competitive market.

On Authentication

I think public-key cryptography authentication will be the only authentication system used in the future, totally replacing user name/password authentication.

Currently we have to keep lots of passwords in mind or in a password manager because every site needs a different one (you can not use the same password on every site since if someone has access to your password at any site he would have access to any of your sites).

Public-key cryptography authentication also allows us to not require the usage of public key infrastructure and be able to have a decentralized web of trust. Privacy-enhanced Electronic Mail will not work in a hierarchy of trust.

To replace user name/password authentication with public-key cryptography authentication in the web the first step would be making it easier to generate and use key pairs in the web browser.

Public-key cryptography authentication also has the advantage of being single sign-on as you only has to type the password of your private key once for all the session.

Cryptographic smart cards are the most secure way of authentication as they keep the private key always on the card and are also password protected.

Van Jacobson is currently leading this Copernican Revolution to make the Internet a Content-centric networking by disassociating the network from the data. He explains it in A New Way to look at Networking.

2008-07-25

On Server side WEB Application Frameworks

Server side web application frameworks are a dead end.

Now that most mobile phones have good JavaScript support (mostly by the use of WebKit in them) and are not just dumb terminals, it is time to develop only client side web applications and stop using the page-per-request web application model.

The future is REST (to let web crawlers index the web) or maybe other Service-oriented architecture protocol and XML or JSON for the data.

The main problem with server side web application frameworks are the load that impose HTML generation on the server and also the long waits to process an action that maybe the server does not need to deal with.

Struts (both Struts1 and Struts2 aka WebWork) have a problem with its design since although there is a controller that forwards actions to views you already need to know what view you are going to render since you have to pass the correct beans to it thus breaking the Model View Controller (MVC) architectural pattern.

public class MyAction extends ActionSupport
{
public ActionForward execute(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response) throws Exception
{
...
request.setAttribute("myBean", myBean);
return mapping.findForward("success");
}
}
Just by having to obtain a bean and attach it to the request for the view to render it you are associating a defined view, making mapping.findForward("success"); unneeded as you could just specify the actual view.

Most server side MVC (like Struts and JavaServer Faces) frameworks are just anachronistic by not supporting REST Web Services and need to be replaced by implementing the controller in the client with JavaScript with XHTML for the view and accessing the data model using REST Web Services.

2008-07-15

On Surrogate keys

I am against the usage of surrogate keys when you have a natural key even being it of type VARCHAR.

There should not be a problem using a VARCHAR as a primary key since if the database is correctly architected it may use internally an integer index to join to other tables. And anyway you always have to lookup the surrogate key from the natural key as the surrogate key should never be exposed outside of the application. You still have to declare UNIQUE the natural key and that will hurt the same (or even more), as having it as PRIMARY KEY, on INSERT.

Is there a REAL performance difference between INT and VARCHAR primary keys?

2008-07-13

On Database Persistence

Following the great failure of Java EE 1.4 Enterprise JavaBean 2.x Entity Bean (Bean-Managed Persistence & Container-Managed Persistence) (now deprecated). Please, read Expert One-on-One J2EE Development without EJB to understand why EJB 1.x and 2.x has hurt so much Java EE reputation. Also watch Rod Johnson - Lessons Learned from Java EE to understand the dangers of design by committee.

I do not use any ORM tool in my Java projects as they create more problems than they try to solve. Hibernate vs JPA vs JDO - pros and cons of each?
Object-Relational mapping is the Vietnam of Computer Science
ORM is an anti-pattern
Using an ORM or plain SQL?

If you insist on using an ORM, here are two strong free implementations of the Java EE Java Persistence API (Java Persistence/What is JPA?) to choose from:
What Java ORM do you prefer, and why?
JPA Implementations - Which one is the best to use?

2008-07-05

On task list managers

The use cases I need for a task list application are:
  • I am at the middle of nowhere and some task pops up in my head. I need to save it quickly to prevent it being forgotten. I need to action my mobile phone into "new task" and type the text or record a voice message that will be voice recognized or later transcribed by me to text to be able to full text search it.
  • I also need to quickly append to tasks. So, I need to buy something and I just type shop and my shopping list entry appears. I select it and just add an item to it. (this feature needs full-text search with partial word matching)
  • I need to be able to full text search all my tasks, and since they are so many, they should be indexed to be able to search (like Google) by typing some words. They could be tag categorized (hierarchies are not as useful).
  • Each task should have a reviewed date attached to let me be able to review most old entries and update that date after review.
  • Tasks should notify me at a certain date/time and/or (using Assisted GPS) when I am near a place where I may be able to complete the task (just like Geolife does). To this end each task should have optional date/time and location fields.
  • There are certain repetitive tasks like, for example, cutting the grass that need to be rescheduled once they are completed (otherwise I forget and too much time can pass), so I need to set a time to notify and be able to review if it was too long or too short.
  • Be able to group tasks into toplevel projects or goals. So tasks can have subtasks. And be able to perform operations like review on a task's subtasks.
  • Any task may depend on any other task that would block the first task until it is completed. The user interface should (by default) show only tasks that do not depend on any other for completion. Dependent tasks may be external (maybe other person) and they may be on the shared task list with that other person or even in a public task list.
  • It needs to be scalable to a lot of task entries without losing performance. It should be as O(1) as possible. I have 800 tasks just for one year of use (the Nokia N80 slows a lot with so many tasks) so it would need to be scalable to 100000 tasks to cover a life time. This requirement adds up to using efficient storage like SQLite and having a GUI that can cope with 100000 tasks in the full task list view (probably it should only render some tasks at a time to be fast and do not require huge amounts of memory something that the N80 is not doing)
  • I need to be able to sort tasks by priority. (You can not sort tasks by priority if you have more than 5 tasks using just 5 priorities)
  • Off-line usage. I may not have Internet connection but I still need to access (and search) my tasks.
  • Synchronization. I want to be able to add, edit and remove tasks from by mobile device and my desktop computer. This will also serve as backup as the tasks will reside in any synchronized device.
  • Keep history of changes. In case I did an incorrect edit or deletion.
GTD: Task management with Android software apps is also a good requirements list and a comparison of Android applications.
I am currently using my Nokia N80 as my task list manager, but as you can see from the above list it does not meet all my requirements.
A good on-line task manager is Remember The Milk: Online to do list and task management but it is mostly an online app. Things for Mac and iPhone is also a great one.