JGate Blog

There are many advantages when a site offers OpenID or OAuth for login. Signup is instant and the users don't have to rember a new user name and/or a new password. For additional security an user can opt for two-factor authentication, which is offered by Google's federated login.

When a site accepts passwords, it is responsible to keep them secure. As many users reuse passwords, a security breach can cause broader damage, if a site uses unsalted hashes or wrong hash algorithms.

When using OpenID, there is no need to store any passwords. There are ready-made solutions for nearly every programming language. See: http://wiki.openid.net/w/page/12995176/Libraries

We added openid4java to the AppJet native API so that every AppJet application can now use OpenID and act as an OpenID consumer. The lib-login library can be used as a drop-in replacement for the lib-rpx library which does the same but relies on a third-party service (Jan Rain). Lib-login additionally supports Twitter's OAuth variant for authentication.

For a new web project we decided to go for a template solution as a replacement for AppJet's print tags. The print tags are fine, they ensure well-formed HTML code and are easy to handle. By using them however HTML code tends to get mixed up with application logic, something we didn't want for the new site.

EJS in AppJet 2

AppJet 2 (Etherpad) uses EJS, an engine normally used on the client side. By using AppJet's JavaScript engine these templates can be evaluated at the server side. A great solution.

Closure Templates

After evaluating different templating tools we decided to give Closure Templates a try. Google is making some real strong statements about it:

• Easy to read. You can clearly see the structure of the output HTML from the structure of the template code. Messages for translation are inline for extra readability.
• Designed for programmers. Templates are simply functions that can call each other. The syntax includes constructs familiar to programmers. You can put multiple templates in one source file.
• Battle-tested. Closure Templates are used extensively in some of the largest web applications in the world, including Gmail and Google Docs.

The engine comes in two flavors, a compiler that generates client-side JavaScript and a Java library for server-side use and that one is a perfect fit for AppJet which is running on the JVM.

The Solution

We added the Closure Java library and have built a corresponding JavaScript library named lib-soy. Have a look at it to see how the templates are separated from the code in an extra section called /* appjet:soy */. The lib also includes the original sample from Google's tutorial:

/**
 * Greets a person and optionally a list of other people.
 * @param name The name of the person.
 * @param [additionalNames] The additional names to greet.
 */
{template .helloNames}
  // Greet the person.
  {call .helloName data="all" /}
  // Greet the additional people.
  {foreach $additionalName in $additionalNames}
    {call .helloName}
      {param name: $additionalName /}
    {/call}
    {if not isLast($additionalName)}
        // break after every line except the last
    {/if}
  {ifempty}
    No additional people to greet.
  {/foreach}
{/template}

This example demonstrates some important concepts of Closure Templates. Hierarchical templates, template parameters, iteration and conditional templates. Enough to get started. Just include lib-soy and you are ready to use soy templates in your own apps.

CouchDB is up and running. It's time to Relax!

We have been busy working on the hosting platform to bring it to the next level. AppJet.jar and CouchDB make a great couple and offer a solution for building more "serious" applications. From now on CouchDB will be the main database for AppJet apps, offering some great advantages, technically and for you as a developer:

• Flexible, scalable and increased storage
• JavaScript everywhere (server, database and client)
• A built-in data administration GUI
• A flexible security model for your data
• Support for attachments (css, templates, images etc.)
• Two-way replication for mobile apps, off-line apps and new use cases
• A strong and active community
• Excellent documentation in form of books, presentations, and more
• No lock-in. Take your data and programs where you like.

CouchDB's RESTful interface and JSON format fit nicely into AppJet's programming model. For an example see the couchdb-test app and it's source.

And the best part: Free CouchDB hosting will start very, very soon for every developer on JGate.

Appjet.jar has now a new home on GitHub. We have committed the original version as well as our additions, so that you can see our changes.

We have also prepared a Wiki, a Tracker and a binary distribution of appjet-1.0.4.jar.    

appjet-1.0.4.jar -- Dec 12 2010 20:47:10 GMT+0100

Running an application

$ java -jar appjet-1.0.4.jar myapp.js

That's it! myapp.js should contain the same JavaScript code that you would use for an app hosted on apps.jgate.de.

For full instructions have a look at the wiki.

Today we are announcing a new version of appjet.jar. As you know, AppJet Inc. has focused on other things and there have been no updates to the framework since 2008. Many links of the old jar still point to the appjet.com domain, which has been shut off since Google's takeover. Despite missing images some pages show some strange behavior due to the missing jquery.js library. For instance the callstack cannot be displayed anymore and footer links point to nowhere.

We have corrected all this plus we added an updated RESTful client. You can now use the platform to build RESTful servers and clients with GET, POST, PUT and DELETE methods, like the new CouchDB client. The new client allows it to talk to local and remote CloudDBs, like those offered by Cloudant.

Talking To CouchDB

See the couchdb-test application for details. You can use the public demo db also for you own tests. We will soon offer a private CouchDB addon for every user.

CouchDB offers dozens of new use cases. You can free your data, replicate databases across the Internet, talk to the database backend directly from the browser and many more.

Future Proof

All the new features can not only be used on our hosted platform, we will open source our additions and offer a new AppJet in a Jar distribution so that you can host your own server and use it as a local server on your development machine. Stay tuned.

With the latest change, we added Varnish as a caching reverse proxy. Varnish is often called a "HTTP Accelerator", and it indeed gave our site a huge speed boost.

As soon as the caching layer was activated in week 34, the simultaneous connections to the back-end were reduced by more than 50% and the load was lowered significantly.

Varnish uses numerous techniques to improve the performance. It even has compiled configuration libraries, in-memory logging, and works hand in hand with the OS for memory management.

Cache Invalidation

Recently several sites (for instance Heroku and Posterous) reported the introduction of Varnish, but all have chosen a different cache strategy. The biggest challenge when using caching is the control of the cache TTL and cache invalidation. Varnish has built-in support to control caching, but this requires a lot of work and is specific to Varnish. Posterous has done a more generic, very interesting ajax-based approach. 

We use a more traditional way using parameter control. Varnish only delivers a cached page, when it is absolutely safe to do so. The URL and all parameters have to match for a cache hit. So the backend can control caching by adding a synthetic parameter to every request, that we call a cache token.

To explain this, here is an example: The response to the "list applications" Ajax call is cached for a hour. Without cache control a deleted application, for example, would still be listed for a hour. The trick here is to issue a new cache token with such an operation. Varnish sees a parameter difference and delivers a fresh response from the back-end in return.

Cache TTL Control

Every application that is hosted on JGate, even when running on a custom domain, leverages the caching layer. The time to live, or TTL, can easily be controlled using AppJets response.setCacheable() call. By default, pages are not cached. When set to true, pages are cached for a long time, a perfect choice for pictures and other static content. When set to undefined, the cache headers need to be supplied by the application. This option allows custom control of the cache.

We have been following the Bespin project since its beginning and think that it is the perfect cloud based editor for the AppJet IDE. Since its so called reboot the feature list has dramatically improved. The latest TIP version (0.9a1) includes support for plugins. They were needed to tailor Bespin for its use in the AppJet IDE.

Commands

Beyond basic typing, everything you do in Bespin is controlled by a "command". Commands can be entered on the command line, or invoked by keyboard control. Over time, we expect Bespin to get an additional user interface for working with commands. For now, it's enough to know that commands are how Bespin gets things done. The help command is a useful command to know: it will list all of the commands available to you right now. The Wiki will mention some commands in the context of larger topics, but the up-to-date list of commands is available via help.

Settings

Our plan is to equip the AppJet IDE with a very configurable editor. You can see the list of available settings and change them by using Bespin's set command.

To use Bespin as your editor and to change its default settings go to "My Account". You can choose a white or a dark theme, switch tab sizes, font sizes and other settings.

When stashboard was released as Open Source earlier this week, it immediately caught my attention. This Software is the missing piece for a monitoring and dashboard solution for JGate. The ability to run independently from JGate's hardware makes it a perfect choice. Stashboard uses OAuth to authenticate write access, which is supported by JGate's lib-oauth library. Writing an event is a matter of a few lines of server-side JavaScript. See the source code of the monitoring app, if you are interested in the details.

There are currently three services which are monitored:

• The AppJet Grid, which executes the AppJet apps and provides JavaScript object storage
• The Backup Service, which provides daily backups of all user apps and their data
• The Platform Service, which serves the JGate site and the browser-based IDE

The status application is running on Google AppEngine. If you ever need status information about any of the services running on JGate, use that link.

(download)

Click here to download:

monitoring-and-the-new-stashboard-JpyEdcgzgcHiCbwbfuys.zip (207 KB)

As we announced in our change log, we exchanged the reverse proxy layer from Apache web server to Nginx. There are a lot of positive reports all over the net that favor Nginx over Apache especially for static content and high number of requests. This is due to the different architecture they are using. Apache is using a blocking IO model with threads and processes. Nginx on the other hand is using event loops and a non-blocking model.

The new servers are now running for 40 days with great success. The performance has gone up significantly, not only shown in monitoring values but also in real statistics measured by Google Labs:

Performance overview: On average, pages in your site take 0.7 seconds to load (updated on Jul 5, 2010). This is faster than 96% of sites. These estimates are of low accuracy (fewer than 100 data points). The chart below shows how your site's average page load time has changed over the last few months. For your reference, it also shows the 20th percentile value across all sites, separating slow and fast load times.

Non-blocking IO really seams to be the key to better performance. So it does not surprise that Node.js, which uses the same architecture, is currently getting so much attention. We are keeping an eye on this.