Monthly Archives: June 2013

Create a game and win an all-expenses paid trip to SenchaCon!

bbz10Last month, Sencha and Blackberry sponsored a contest among the top 25 performing Sencha partners from which five would be selected to be demoed at SenchaCon. The goal of the contest was to develop/port apps to the Blackberry Z10 that highlighted the features and benefits of that new device. We were honored to be included in the contest and delighted to find that our entry was named as one of the winners.

Now, as many of my close friends can attest, I have a weakness for contests that dates all the way back to an unfortunate incident that occurred during my senior year of high school. And ever since then, any time a good contest comes along, it captures my full attention. It’s a bit of an unhealthy obsession, actually. This was as true for the first (and last) game of “quarters” that I played back in college as it is today when I’m competing in a hackathon or for a spot in the Inc. 500/5000. And don’t “double-dog dare” me…ever. You’ll end up licking Bourbon Street. But, I digress.

Victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win.

-Sun Tsu

So, the first step in competing was strategizing about what kind of app might make it to the “final five.” I wanted to do something that hadn’t really been attempted before. So, I made a list of key features:

  • It had to be “cool.”
  • It had demonstrate the superior performance of the Blackberry Z10.
  • It had to stretch the limits of what was possible with Sencha Touch and mobile platforms.
  • It had to make our brains hurt (something new that we hadn’t built before).
  • It had to contain at least one Mel Brooks or Monty Python reference.
  • It had to “suck up” to the judges (lesson learned in high school)

So I decided that we’d build a game. Not any game, mind you. THE GAME. The game that initially piqued my interest in computers, way back in 1980 – STAR RAIDERS, running on an Atari 400. Surely a game that had been built nearly 35 years ago to run within 8KB of RAM and perform well on a 4.77 Mhz CPU could be redeveloped to run as an HTML5 app on a mobile phone, right? What could possibly go wrong?!

Creating the Proof of Concept

With the plan in hand, we built out a proof-of-concept just to see if we could get the most basic UI elements to operate efficiently on the platform.  We had to get directional starfield working, as well as some of the visuals for basic game features like “raising the shields” and changing direction of the ship. We also wanted to see if we animate the firing of photon torpedoes to be reasonably performant. Fortunately, the overall HTML5 performance of the Blackberry Z10 was really great. This is a device that definitely merits your consideration the next time that you find yourself shopping for a smartphone.

My God, It’s Full of Stars

stars

As it turned out, the starfield was actually relatively easy to implement- largely because we could “cheat” and port this HTML 5 example over to a Sencha Touch view. One of the challenges, however, is that since the starfield was built for an HTML5 canvas, we had to ensure that Sencha’s Ext.draw.Component class would always generate a <canvas> tag, regardless of the device that the app was running on (currently Sencha Touch outputs <canvas> on iOS and SVG for everything else). The solution was fairly straightforward, albeit not particularly well documented. Sencha Touch enables you to explicitly set the drawing engine as illustrated by the following code snippet:

Ext.define('MyApp.view.StarCanvas', {
	extend: 'Ext.draw.Component',
	alias: 'widget.starcanvas',
	engine: 'Ext.draw.engine.Canvas'
});

The rest of the starfield was, more or less, a direct port. Of course, we had to fit it into the Sencha framework and made some minor mods to support an “aft view” as well as tweaked the code to randomize the star’s starting positions:

Ext.define('MyApp.view.ViewScreen', {
	extend: 'Ext.Container',
	alias: 'widget.viewscreen',
	requires: [
			'Ext.draw.Component',
			'MyApp.view.StarCanvas'
	],

	c_x: null,
	c_y: null,
	starTimer: null,

	config: {
		itemId: 'viewscreen',
		style: 'background-color: #000000',
		layout: {
			type: 'fit'
		},
		aftView: false
	},

	onViewscreenActivate: function() {
		this.onChangeOrientation();
		this.initializeStars();
		this.doAnimate();

		//console.log(Ext.ComponentQuery.query("#sndThrust")[0].getDuration());
		/*
        Ext.device.Orientation.on({
            orientationchange: {
                fn: this.onOrientationChange,
                scope: this
            }
        });
        */
	},

	updateAftView: function(nVal) {
		this.initializeStars();
		this.star_speed *= -1;
	},

	initialize: function() {
		this.n = 512;
		this.w = 0;
		this.h = 0;
		this.x = 0;
		this.y = 0;
		this.z = 0;
		this.star_color_ratio = 1;
		this.star_ratio = 256;
		this.star_speed = 1;
		this.star_speed_save = 0;
		this.star = new Array(this.n);
		this.fps = 0;
		this.cursor_x = 0;
		this.cursor_y = 0;
		this.canvas_x = 0;
		this.canvas_y = 0;
		this.setItems({
			xtype: 'starcanvas',
			itemId: 'viewscreenCanvas'
		});
		this.canvas = this.down('#viewscreenCanvas');
		this.domCanvas = Ext.get(this.canvas.getSurface().element.query('canvas')[0]);
		this.on({
			resize: this.onChangeOrientation,
			scope: this
		});
		Ext.Function.defer(this.onViewscreenActivate, 750, this);
		this.animateStarField = Ext.Function.bind(this.doAnimate, this);
		this.callParent(arguments);
	},

	onOrientationChange: function(e) {
		e = e[0];

		// var s = this.down('starcanvas');
		// console.log('g', e.gamma);
		// console.log('b', e.beta);

		this.cursor_x += e.gamma - this.c_x;
		this.c_x = e.gamma;
		this.cursor_y += e.beta - this.c_y;
		this.c_y = e.beta;
	},

	onChangeOrientation: function() {
		var el = Ext.get(this.element.select('canvas').elements[0]);
		this.w = el.getWidth();
		this.h = el.getHeight();

		this.x = Math.round(this.w / 2);
		this.y = Math.round(this.h / 2);
		this.z = (this.w + this.h) / 2;
		this.star_color_ratio = 1 / this.z;
		this.cursor_x = this.x;
		this.cursor_y = this.y;
	},

	doAnimate: function() {
		context = this.domCanvas.dom.getContext('2d');
		context.fillStyle = 'rgb(0,0,0)';
		context.strokeStyle = 'rgb(255,255,255)';
		mouse_x = this.cursor_x - this.x;
		mouse_y = this.cursor_y - this.y;
		// console.log(mouse_x, mouse_y);
		// var path = '';
		// this.canvas.removeAll(true);
		context.fillRect(0, 0, this.w, this.h);
		for (var i = 0; i < this.n; i++) {
			test = true;
			this.star_x_save = this.star[i][3];
			this.star_y_save = this.star[i][4];
			this.star[i][0] += mouse_x >> 4;
			if (this.star[i][0] > this.x << 1) {
				this.star[i][0] -= this.w << 1;
				test = false;
			}
			if (this.star[i][0] < -this.x << 1) {
				this.star[i][0] += this.w << 1;
				test = false;
			}
			this.star[i][1] += mouse_y >> 4;
			if (this.star[i][1] > this.y << 1) {
				this.star[i][1] -= this.h << 1;
				test = false;
			}
			if (this.star[i][1] < -this.y << 1) {
				this.star[i][1] += this.h << 1;
				test = false;
			}
			this.star[i][2] -= this.star_speed;
			if (this.star[i][2] > this.z) {
				this.star[i][2] -= this.z;
				test = false;
			}
			if (this.star[i][2] < 0) {
				this.star[i][2] += this.z;
				test = false;
			}
			if (!test) {
				// reset starting position and angle
				this.star[i][0] = Math.random() * this.w * 2 - this.x * 2;
				this.star[i][1] = Math.random() * this.h * 2 - this.y * 2;
				this.star[i][2] = Math.round(Math.random() * this.z);
			}
			this.star[i][3] = this.x + (this.star[i][0] / this.star[i][2]) * this.star_ratio;
			this.star[i][4] = this.y + (this.star[i][1] / this.star[i][2]) * this.star_ratio;
			if (this.star_x_save > 0 && this.star_x_save < this.w && this.star_y_save > 0 && this.star_y_save < this.h && test) {
				context.lineWidth = (1 - this.star_color_ratio * this.star[i][2]) * 2;
				context.beginPath();
				context.moveTo(this.star_x_save, this.star_y_save);
				context.lineTo(this.star[i][3], this.star[i][4]);
				context.stroke();
				context.closePath();
			}
		}
		// surface.renderFrame();
		// update playfield
		requestAnimationFrame(this.animateStarField);
	},

	initializeStars: function() {
		for (var i = 0; i < this.n; i++) {
			this.star[i] = new Array(5);
			this.star[i][0] = Math.random() * this.w * 2 - this.x * 2;
			this.star[i][1] = Math.random() * this.h * 2 - this.y * 2;
			this.star[i][2] = Math.round(Math.random() * this.z);
			this.star[i][3] = 0;
			this.star[i][4] = 0;
		}
	}
});

As you can see from the code snippet, we toyed around with using the device’s accelerometer to control ship/starfield movement but ultimately abandoned the idea as it was too difficult for my forty-four year old fingers to accurately control.

In Space, No One Can Hear the Developer Scream

Another somewhat unexpected challenge involved adding sound effects to the app. I had heard that HTML5 audio had some “issues” but was completely unprepared for the truly heinous support (or lack thereof) that is currently out there. This was 2013, after all, and I figured that a full five years after the introduction of the iPhone that surely browser support for MP3 playback must have been troubleshot. I could not have been more wrong.

Our initial “brute-force” implementation for adding sound was to simply add hidden Ext.Audio instances to a view class as illustrated below:

 items:[
  { xtype: 'audio',
   url  : 'resources/sounds/pewpew.mp3',
   itemId: 'sndFirePhoton',
   hidden: true
  },
  {
   xtype: 'audio',
   url  : 'resources/sounds/thrust.mp3',
   itemId: 'sndEngineThrust',
   loop: true,
   hidden: true
  }
 ]

We then selectively executed the Ext.audio.Play() method based on the type of operation that was being performed by the user. And this worked great…until we tested it on mobile devices where it failed in strange and magnificent ways! It turns out that iOS has some significant issues with playing back two audio streams simultaneously, and the Blackberry OS would randomly fail to load our MP3 files once a certain memory usage threshold was reached.

Pull-yourself-together! “What will you do?” Is this a question? You will show him you remember that he is Mr. Incredible, and you will remind him who *you* are. Well, you know where he is. Go, confront the problem. Fight! Win!

Edna Mode

Ultimately, our stop-gap solution was to produce a sound “sprite.” Essentially we merged all of the game’s sound effects into a single file (cudos to Adobe Audition) and then extended the Ext.Audio class to play segments on an as-needed basis:

Ext.define('MyApp.view.Sounds', {
    extend: 'Ext.Audio',
    xtype: 'sounds',

    audioSprite: null, // DOM reference
    audioQueue: [],
    canPlayThrough: false,

    config: {
        hidden: true,

        ambientTrack: 'sndThrust',
        playAmbient: false,

        tracks: {
            sndOn: {
                start: 0,
                length: 0.35
            },
            sndOff: {
                start: 1.38,
                length: 0.5
            },
            sndWarp: {
                start: 2.974,
                length: 6.456
            },
            sndWarpExit: {
                start: 9.43,
                length: 6.96
            },
            sndTorp: {
                start: 17.39,
                length: 1.32
            }
        },

        url: 'resources/sounds/audiosprite.mp3',
        itemId: 'sndSprite'
    },

    initialize: function() {
        this.callParent(arguments);
        this.audioSprite = Ext.get(this.element.select('audio').elements[0]);
        this.audioTimeUpdate = Ext.Function.bind(this.onTimeUpdate, this);
    },

    onTimeUpdate: function(e) {

        if (this.audioQueue.length > 0) {
            if (this.getCurrentTime() >= this.audioQueue[0].start + this.audioQueue[0].length) {
                this.pause();
                Ext.Array.erase(this.audioQueue, 0, 1);
                if (this.audioQueue.length > 0)
                    this.playQueue();
                else if (this.getAmbientTrack() != '' && this.getPlayAmbient()) {
                    this.playTrack(this.getAmbientTrack());
                }
            }
            requestAnimationFrame(this.audioTimeUpdate);
        }

    },

    updatePlayAmbient: function(bool) {
        if (bool) {
            this.playTrack(this.getAmbientTrack());
        }
    },

    playQueue: function(bNow, err) {

        try {
            this.setCurrentTime(this.audioQueue[0].start);
            this.play();
            requestAnimationFrame(this.audioTimeUpdate);
        } catch (e) {
            // overcome potential ios seeking issue
            this.play();
            this.pause();
        }

    },

    playTrack: function(name, bNow) {

        if (bNow) {
            this.audioQueue = [];
        }
        this.audioQueue.push(
            this.getTracks()[name]);

        this.playQueue(bNow);

    }
})

This effectively solved our file loading issue on Blackberry and iOS started playing back a bit more reliably, but is still a bit dodgy. It also had the side benefit of reducing the number of http requests to sound files, and anything that you can do to reduce the number of http transactions is helpful for performance. Ultimately, I’ll refactor this again to try and make it even more stable, but it’s borderline acceptable for a 1.0 release of a “demo app”

Always remember to bring the funny

For me, the best part of working on this project was finding interesting ways to inject geek humor into the proceedings

Like engaging the warp drive…

ludicrous

…and putting a ridiculous backstory into a “Star Wars” crawl  using CSS3…

crawl

…and making an oblique reference to the lead developer of Sencha Touch (Jacky Nguyen)…

instructions

… and simulating a “cracked” viewscreen when you get caught with your shields down…

cracked

…and designing a scenario where legions of  jQuery developers are out to assimilate you …

jq

Was it over when the Germans bombed Pearl Harbor?

I suppose that what I’ve learned over the years is that it takes just a little bit of insanity and no small amount of effort to win these contests. Reach for the stars, but be prepared when the universe responds by firing a photon in your general direction.

Work on the app continues…we’re still hacking away, posting new builds every few days, optimizing the codebase, and tweaking gameplay at http://webapps.figleaf.com/sr. Because, let me tell you, MATH IS HARD! We’ll have our initial 1.0 release out before SenchaCon on July 16th which will coincide with a full release of the sourcecode on GitHub for everyone’s mutual enjoyment.

Update:

Full sourcecode is available at:
https://github.com/sdruckerfig

 

How to Win Prizes at a Hackathon with Sencha Touch 2.2 Theming

A couple of weeks ago myself and one of my fellow Figs, Jason Perry, attended a MoDevUX Hackathon. Honestly, I had no idea what to expect – the last programming competition that I was in was when I led my high school team to a 12th place finish among all high schools in the area as we typed fast and furiously on an Apple IIE, programming in BASIC.

Obviously, the technology has improved just a little bit since then – as has the competition. After all, several of the hackathon contestants were just a glimmer in their parent’s eyes back in 1987 when going “mobile” meant lugging 20 lbs of equipment around with a handtruck. But I digress…

The goal of the hackathon was to produce a mobile app that had something to do with finance. (Capital One was the big sponsor). You got bonus points for developing apps for Microsoft Windows phones (Nokia was a sponsor), as well as using various APIs (Mashery.com) and/or adding VOIP/Voice/SMS communications to an app using the awesome Twilio API.

walletA third team member, whom we met at the Hackathon, came up with the idea to produce a mobile/social app that would enable people to exchange international currencies at the airport, without having to pay the ridiculous transaction fees charged by the on-site money exchanges.

My strategy was to cover as many bases as possible to ensure that we’d win *something* – I figured a first-time win for Team Fig would go a long way towards boosting company morale (and I wanted a Microsoft phone to complete our suite of  test devices). So, with the immortal words of Sean Connery ringing in my ear, we set off to design and build an app that would support the following features:

1) Look and feel like a native Windows Mobile phone (as well as be compatible across a broad range of other mobile devices)
2) Integrate voice/sms features from Twilio
3) Calculate currency conversion based on the current exchange rates via a third-party API
4) Link to a back-office database that would help facilitate matching compatible requests.
5) Translate requests and messages into different languages via the Google Translate API

And get functional prototype programmed and tested within a seven hour timeframe on a device (Nokia Windows Phone) that we had no prior development experience with. What could possibly go wrong?

Divide and Conquer

In order to make the deadline, we decided to use Sencha Touch 2.2 as our underlying development platform as they had just begun supporting Windows Mobile devices and had a Windows-Mobile compatible theme ready to go “out of the box.” I was responsible for developing the “front-end” Touch UI, while Jason was responsible for standing up a publicly accessible XAMPP stack and all the “back-end” API integration (PHP, figuring out how to use the Twilio API, etc).

Supporting Windows Mobile Devices

Supporting Windows Mobile 8 was the lynchpin of our strategy. I figured that since it was relatively new, most hackathon participants wouldn’t know how to develop for it. And even if they were familiar with it, they probably wouldn’t be able to throw an app together with Microsoft’s native tools as quickly as we could by using Sencha Touch.

Supporting Windows Mobile in Sencha Touch really comes down to linking to the “out of the box” Windows mobile stylesheet. In a Sencha Touch project, this can be accomplished by just typing a few lines out into the app.json file, which controls the loading of stylesheets based on device platform:

"css": [
 {
  "platform": ['ie10'],
  "theme": "Windows",
  "update": "delta",
  "path": "resources/css/ie.css"
 },
 {
  "path": "resources/css/bb.css",
  "platform": ['blackberry'],
  "theme": "Blackberry",
  "update": "delta"
 },
 {
   "path": "resources/css/default.css",
   "platform": ['chrome', 'safari', 'ios'],
   "theme": "Default",
   "update": "delta"
 }
]

The CSS file itself is generated by SASS/Compass, from a .SCSS file that resembled the following:

@import ‘sencha-touch/windows’;
@import ‘sencha-touch/windows/all’;

@include icon(‘upload’);
@include icon(‘left’);

So, with about 10 lines of code, we were able to leverage years of Sencha Touch development experience to produce an app for a device that we had never actually laid our hands on. We could also use Sencha Architect to rapidly prototype the UI, which was an added bonus!

Integrating Currency Conversion

Integrating currency conversion into an application is a pretty straightforward affair. We used a free currency rate service that supported a JSON-P interface that could be called directly from our Sencha Touch app:

Ext.define('MyApp.util.Currency', {
 statics: {
  conversionRate: function( from, to, callback, scope) {
   Ext.data.JsonP.request({
     url: 'http://rate-exchange.appspot.com/currency',
     params: {
      from: from,
      to: to
     },
     success: function(response) {
	callback.call(scope, response);
     }
   });
  }
 }
});

In this case, from and to represent currency codes (e.g. USD is US Dollars).

Using the Google Translate API

One of the features that we added to the app was the ability to send SMS and voice-synthesized phone messages (via Twilio), translated into the language of the recipient. Google Translate, while far from perfect, is quite easy to use – albeit not a free service. Integrating it into Sencha Touch was a snap:

Ext.define('MyApp.util.Translate', {
 statics: {
  key: 'AIza...',
  translate: function(text, source, target, callback, scope) {
   if (source == target) {
    callback.call(scope,text);
   } else {
    Ext.data.JsonP.request({
      url: 'https://www.googleapis.com/language/translate/v2',
      params: {
	key: MyApp.util.Translate.key,
	source: source,
	target: target,
	q: text
      },
      success: function(response) {
        callback.call(scope, response.data.translations[0].translatedText);
      }
    });
   }
  }
 }
});

Adding tel/sms services using Twilio

Twilio is a cloud-based service that enables you to create apps which send/receive/route phone calls, record audio, capture stats, send SMS messages, and more. They’ve even got an HTML5 SDK and series of REST services that you can invoke from Sencha Touch. We designed the app so that a user could message someone via SMS and voice-synthesized message (in language) who had a reciprocal currency exchange need at the designated airport, while keeping the user’s telephone numbers private.

In order to accomplish this, we defined a static class in our app that invoked a couple of custom webservices that we developed initially in PHP, and then later ported over to ColdFusion:

Ext.define('MyApp.util.Twilio', {
 statics: {
  sendText: function( transactionId, msg, callback, scope) {
    Ext.Ajax.request({
      url: 'http://webapps.figleaf.com/wallet/transaction.cfc?method=sendText',
      params: {
	id: transactionId,
	msg: msg
      },
      success: function(response) {
	callback.call(scope, response);
      }
    });
  },

  sendCall: function( transactionId, msg, callback, scope) {
    Ext.Ajax.request({
      url: 'http://webapps.figleaf.com/wallet/transaction.cfc?method=sendCall',
      params: {
	id: transactionId,
	msg: msg
      },
      success: function(response) {
	callback.call(scope, response);
      }
   });
  }
 }
});

On the server-side, we lookup transaction details and then invoke Twilio to send SMS messages and make the phone call. The way that Twilio scripted phone calls work is that you pass a callback URL that generates “TwiML” to their service. TwiML is quite powerful, but for our limited proof of concept, all we needed to do was tell it to “read” from a script that we translated into language using the Google Translate API.

<!--- send an sms text message --->
<cffunction name="sendText" access="remote" returntype="string" returnformat="plain">
 <cfargument name="id" type="numeric" required="yes">
 <cfargument name="msg" type="string" required="Yes">

 <cfset var q = "">
 <cfset var stSuccess={}>
 <cfquery name="q">
  select * from transaction where transactionid=<cfqueryparam cfsqltype="cf_sql_numeric" value="#arguments.id#">
 </cfquery>
 <cfhttp
   username="[twilio user id]"
   password="[twilio password]"
   url="https://api.twilio.com/2010-04-01/Accounts/[twilio account id]/SMS/Messages" method="post">
     <cfhttpparam name="From" value="[twilio phone num]" type="formfield">
     <cfhttpparam name="To" value="#q.phone#" type="formfield">
     <cfhttpparam name="Body" value="#arguments.msg#" type="formfield">
 </cfhttp>
 <cflog file="twilio" text="#cfhttp.FileContent#">
 <cfset stSuccess= {success: true}>
 <cfreturn serializeJson(stSuccess)>
</cffunction>

<!--- generate TwiMl --->
<cffunction name="playMsg" access="remote" returntype="xml" returnformat="plain">
  <cfargument name="id" type="numeric" required="yes">
  <cfargument name="msg" type="string" required="yes">
  <cfset var q = "">
  <cfset var out = "">
  <cfset var lang="">

  <cfquery name="q">
    select *
    from transaction
    where transactionid=<cfqueryparam cfsqltype="cf_sql_numeric" value="#arguments.id#">
  </cfquery>
  <cfset lang = q.nativeLanguage>
  <cfif listfind("en,es,fr,it,de",lang) is 0>
    <cfset lang="en-gb">
  </cfif>
  <cfxml variable="twiml">
    <cfoutput>
     <Response>
       <Say loop="0" voice="woman" language="#lang#">
          #xmlFormat(arguments.msg)#
       </Say>
     </Response>
    </cfoutput>
 </cfxml>
 <cfreturn twiml>
</cffunction>
<!--- call the user and read from a TwiML generated script --->
<cffunction name="sendCall" access="remote" returntype="string" returnformat="plain">
   <cfargument name="id" type="numeric" required="yes">
   <cfargument name="msg" type="string" required="Yes">
   <cfset var q = "">
   <cfset var stSuccess={}>

   <cfquery name="q">
     select *
     from transaction
     where transactionid=<cfqueryparam cfsqltype="cf_sql_numeric" value="#arguments.id#">
   </cfquery>

    <cfhttp
      username="[twilio user id]"
      password="[twilio password]"
      url="https://api.twilio.com/2010-04-01/Accounts/[twilio account id]/Calls" method="post">
       <cfhttpparam name="From" value="+15045951048" type="formfield">
       <cfhttpparam name="To" value="#q.phone#" type="formfield">
       <cfhttpparam name="Url" value="http://webapps.figleaf.com/wallet/transaction.cfc?method=playMsg&id=#arguments.id#&msg=#urlencodedformat(arguments.msg)#" type="formfield">
    </cfhttp>
    <cflog file="twilio" text="#cfhttp.FileContent#">
    <cfset stSuccess= {success: true}>
    <cfreturn serializeJson(stSuccess)>
 </cffunction>

To the Victor goes the Spoils

shinySo we got the app produced and debugged to the point where we could demonstrate all of the core functionality within a frenetic 6 hour period. After our allotted two minute (!) presentation that seemingly went by in about two seconds, we sat down and waited for the results. The folks from Nokia were kind enough to award us a new phone and nice tote bag for our efforts. I was happy to add a Microsoft device to Fig Leaf’s burgeoning collection of mobile test devices.

And then I went home and passed out for four hours from exhaustion.

You can try out the app at http://webapps.figleaf.com/wallet on your mobile device or by using Google Chrome on your PC (note that the Twilio features are disabled as we’re using a test account). Not too bad for 6 hours of frantic coding!