Category: Programming

Getting Started: With ember.js 2 + firebase + materialize css.

Introduction

This article assumes that you have some idea about programming and web development in general. I’ll still try to be as descriptive as possible.

The project associated with this article is on GitHub!
If you want to try it out, follow these instructions:

First, make sure you have the first three tools in “Part I – Acquiring the tools”.

Then, go to a folder where you would want to store the cloned project. Then, do this:

git clone https://github.com/jayantbh/shoutout.git shoutout_clone
cd shoutout_clone
git reset --hard 12289dc050d60156804e58cede11e73db96bbb04
npm install
bower install
ember server

This project is also available for viewing live on https://shoutoutdb.firebaseapp.com/.

Part I – Acquiring the tools

Tools you’ll need and how to install them:
Note: I’m assuming you have an idea of how node and npm works. Even it not, won’t really matter too much.

  • node.js and npm
    • Installing npm from a simple apt-get install gives you the v10.x.x of node.js. I, or anyone else will suggest installing the following way:
      • sudo apt-get install curl
        curl -sL https://deb.nodesource.com/setup_0.12 | sudo bash -
        sudo apt-get install nodejs
        
  • bower
    • npm install -g bower
  • ember-cli
    • npm install -g ember-cli
  • emberfire
    • emberfire is a Firebase adapter for ember.js, don’t worry about it, we’ll discuss it later in the article. This must be installed after ember-cli has been installed.
      • ember install emberfire
  • phantom.js 2 (kinda optional)
    • Ember uses Phantom for running automated tests. But since we’re not gonna talk about running tests here, we won’t really be using it. In case ember throws up errors saying it needs phantom.js, you can install this, this way. Phantom.js 1 won’t really work here.
      • npm install -g phantomjs2
  • materialize css (optional)
    • Materialize CSS is a front-end design framework based on Google’s Material Design. I’ll talk about this wherever needed as I’ve used this in the project. If you’re more familiar with Twitter Bootstrap, you should be able to change most of this code into Bootstrap with minimal effort.
      • npm install -g ember-cli-materialize --save
        ember g ember-cli-materialize
  • js cookie (optional)
    • It is a pure JS cookie utility library. We’re using this in the project, so if you want to follow exactly as I go, you will need to use this when needed.
      • Obtain from THIS link.
  • ember inspector, chrome/firefox extension (optional, but very useful)
    • This is just like the chrome/firefox/etc inspector that you’re familiar with, but for ember.js apps. It shows you all the routes you’re using, and available to you, all the data in your ember app, ember element nesting, and tons of other stuff.

Part II – Getting started

Now that you’ve acquired pretty much all the tools needed to get started, let’s talk about how you’d move ahead.
Note: I’m using ubuntu, even then, all ember related commands will remain the same.

Initialize the project

Go to the folder you intend to generate your project in. For the sake of similarity with my existing code, we’ll call the project name as “ShoutOut”.
So my intended project folder will be “shoutout”. We use all lowercase because some tools have problems with uppercase letters.

jayantbhawal@px:~$ cd Projects/
jayantbhawal@px:~/Projects$ ember new shoutout

The command: ember new shoutout creates a new folder named “shoutout” and generates the default ember app structure with files inside it.
You’ll see a ton of files and folders being created and some npm installs, and then bower installs. You now have to navigate into the project folder, and run the server.

jayantbhawal@px:~/Projects$ cd shoutout
jayantbhawal@px:~/Projects/shoutout$ ember server

This automatically starts a server at the default port 4200. Your project is now viewable at http://localhost:4200
To change the port at which you want to start the server at, do this, and you can replace 1234 with any number under 65535 and over 1000, to be safe:

ember server --port 1234

OR

ember s --port 1234

Note: “s” is an alias for “server” in ember-cli.

You should be seeing one piece of text in big letters saying Welcome to Ember.js”. This piece of text comes from a Handlebars template file at “app/templates/application.hbs“.

If you haven’t already, now would be a great time to fire up your code editor and load the project.

My recommended code editors in order of preference are:

  1. WebStorm 10
  2. Sublime Text 3
  3. Notepad++

Important Note: If you don’t know how to configure WebStorm at all, you may want to stick to Sublime Text 3. I may add the required configuration for WS10 later.

Now if you’ve launched the editor, let’s deal with some odd-jobs and tiny annoyances. Listing them:

  1. Open “shoutout/config/environment.js”, find a line saying,
    firebase: 'https://[YOUR-FIREBASEIO-URL].firebaseio.com/',

    to this

    firebase: 'https://shoutoutdb.firebaseio.com/',

    or leave it as it is now, we’ll get back to it later.

  2. Open “shoutout/package.json”, and remove a line that has:
    "ember-cli-content-security-policy":"[some-version]"

    The text may not be exactly same, but you’ll get it. Remove the whole line.

  3. Go to “shoutout/public”, create a folder named “public”, create a subfolder named “js” or whatever you prefer to store your javascript files in, and store this file there.
  4. Now, restart your server. To this, use CTRL+C to stop the running server, and start the server again. You should notice that if you were noticing some “content security policy violation” errors or messages before, they’re gone, which is genuinely a relief.

One last thing to explain before we start, the meanings of the directory structure of your app, and some guide regarding how to use the Ember-CLI, courtesy of Ember-CLI user guide.

Folder Layout

File/directory Purpose
app/ Contains your Ember application’s code. Javascript files in this directory are compiled through the ES6 module transpiler and concatenated into a file called .js. See the table below for more details.
dist/ Contains the distributable (optimized and self-contained) output of your application. Deploy this to your server!
public/ This directory will be copied verbatim into the root of your built application. Use this for assets that don’t have a build step, such as images or fonts.
tests/ Includes your app’s unit and integration tests, as well as various helpers to load and run the tests.
tmp/ Temporary application build-step and debug output.
bower_components/ Bower dependencies (both default and user-installed).
node_modules/ npm dependencies (both default and user-installed).
vendor/ Your external dependencies not installed with Bower or npm.
.jshintrc JSHint configuration.
.gitignore Git configuration for ignored files.
ember-cli-build.js Contains the build specification for Broccoli.
bower.json Bower configuration and dependency list. See Managing Dependencies.
package.json npm configuration and dependency list. Mainly used to list the dependencies needed for asset compilation.

Layout within app directory

File/directory Purpose
app/app.js Your application’s entry point. This is the first executed module.
app/index.html The only page of your single-page app! Includes dependencies, and kickstarts your Ember application. See app/index.html.
app/router.js Your route configuration. The routes defined here correspond to routes in app/routes/.
app/styles/ Contains your stylesheets, whether SASS, LESS, Stylus, Compass, or plain CSS (though only one type is allowed, see Asset Compilation). These are all compiled into .css.
app/templates/ Your HTMLBars templates. These are compiled to /dist/assets/.js. The templates are named the same as their filename, minus the extension (i.e.templates/foo/bar.hbs -> foo/bar).
app/controllers/,app/models/, etc. Modules resolved by the Ember CLI resolver. See Using Modules & the Resolver.

app/index.html

The app/index.html file lays the foundation for your application. This is where the basic DOM structure is laid out, the title attribute is set, and stylesheet/javascript includes are done. In addition to this, app/index.html includes multiple hooks – {{content-for 'head'}} and {{content-for 'body'}} – that can be used by addons to inject content into your application’s head or body. These hooks need to be left in place for your application to function properly however, they can be safely ignored unless you are directly working with a particular addon.

Using Ember CLI

Command Purpose
ember Prints out a list of available commands.
ember new Creates a directory called and in it, generates an application structure. If git is available the directory will be initialized as a git repository and an initial commit will be created. Use --skip-git flag to disable this feature.
ember init Generates an application structure in the current directory.
ember build Builds the application into the dist/ directory (customize via the --output-path flag). Use the --environment flag to specify the build environment (defaults to development). Use the --watch flag to keep the process running and rebuilding when changes occur.
ember server Starts the server. The default port is 4200. Use the --proxy flag to proxy all ajax requests to the given address. For example,ember server --proxy http://127.0.0.1:8080 will proxy all ajax requests to the server running at http://127.0.0.1:8080.
ember generate Runs a specific generator. To see available generators, run ember help generate.
ember destroy Removes code created by the generate command. If the code was generated with the --pod flag, you must use the same flag when running thedestroy command.
ember test Run tests with Testem in CI mode. You can pass any options to Testem through a testem.json file. By default, Ember CLI will search for it under your project’s root. Alternatively, you can specify a config-file.
ember install Installs the given addon into your project and saves it to the package.json file. If provided, the command will run the addon’s default blueprint.

 

You may also want to know how Handlebars templates are used.

And if you feel a bit lost, you can look around the official emberjs user-guide.

You’re done. Time to start coding our application.

Part III – Diving into the code

You are about to build a simple ember app that allows users to post on a public feed.
The goal is to let them create a user profile that will be stored locally in the browser cookies, and let them post stuff on the feed.

We intend to make a single page app, where the major application logic works under one route.
Note: If you’re confused by any term, don’t worry, follow along, you’ll understand.

Section A – Configuring Routes

Let’s start with creating two routes.

ember generate route index
ember generate route profile

You can also replace “generate” with “g”.

You should see something like this, for once each for both the commands.

jayantbhawal@px:~/Projects/shoutout$ ember g route index
version: 1.13.8
Could not find watchman, falling back to NodeWatcher for file system events.
Visit http://www.ember-cli.com/user-guide/#watchman for more info.
installing route
create app/routes/index.js
create app/templates/index.hbs
updating router
add route index
installing route-index
create tests/unit/routes/index-test.js

For the attentive ones, you see version: 1.13.8 although the article mentions version 2. That is the version that comes with the latest ember-cli, and even according to it’s developers, it’s just emberjs 2 in disguise.

Now open app/router.js.

import Ember from 'ember';
import config from './config/environment';

var Router = Ember.Router.extend({
  location: config.locationType
});

Router.map(function() {
  this.route('index', {path:'/'});
  this.route('profile', {});
});

export default Router;

It should look like this, except the {path:’/’} in the index route declaration. You have to add that.
That tells the app to load anything related to the index route when the user loads localhost:4200 (or whatever your port is).
We did not pass any path parameter to the profile route, so by default it’ll try to load anything related to the profile route on the path: ‘/profile’.

Section B – Working on the ‘application’ template

The app/templates/application.hbs template has stuff that needs to be persistent throughout the app. You can add things such as a Navigation Bar or a site-wide footer here.

Your application.hbs must be looking nearly empty, in my case, I’ve added a navbar built with Materialize.

Open app/templates/application.hbs.

<nav>
    <div class="nav-wrapper">
      <div style="margin-left: 20px;">{{#link-to 'index' class='brand-logo'}}ShoutOut! <span class="grey-text text-lighten-2" style="font-size: 0.5em;">v0.3a</span>{{/link-to}}</div>
      <ul id="nav-mobile" class="right">
         <li>{{#link-to 'profile'}}Profile{{/link-to}}</li>
      </ul>
   </div>
</nav>
<div class="container">{{outlet}}</div>

{{!-- ember-cli-materialize modal container --}}
{{md-modal-container}}

New stuff!

  1. {{#link-to ‘route-name’ …additional parameters…}} someText {{/link-to}}
    This is called a link-to helper. Helpers are things in ember.js that “help” you perform certain actions. The link-to helper helps you to switch the route, and hence the view of the page without having to reload the entire app.
  2. {{outlet}}
    While not exactly useful here, the outlet component loads any data from any nested routes. Our app won’t be using nested routes because we don’t need to.
    Note that the rest of the entire application is going to be rendered in the {{outlet}} above. Reasons is described below.
    Either way, here is some information about nested routes and how outlet works, that you can skip for now if you want to focus on building the app first.

    1. Nested Routes
      If you had a posts route, you would expect it to load all posts. If you had a posts/1, posts/2 … route, you would expect these routes to load posts with IDs 1, 2, … .
      Now, assume that your app is divided into two vertical parts. The left part has a list of post titles. The right shows the content of whatever post title is clicked. In this scenario, the content in your app/template/posts.hbs would be consisting of mostly the left part. The right part would only have one thing. That is…
    2. Outlet
      The outlet component shows content from its child route./posts/1 is a child route of /posts. So when /posts/1 loads the data of only the post, that data is shown nicely fitting inside the parent route of /posts.
    3. Generating Child Routes
      Child routes can be generated with ember-cli this way:

      ember g route posts/post
  3. {{md-modal-container}}
    That is a result of using ember-cli-materialize, and won’t be there otherwise. It’s used for modals in Materialize. Interesting stuff.
  4. {{!– … –}}
    This is how comments are used in Handlebars.

Section C – Working on the ‘index’ route.

We’re going to have two main things on our index page. One post input area, one post feed area. It’s a good practice to break a big page into smaller components in handlebars.

Note: You’ll be needing quite a few components now, to generate components using ember-cli, do this:

ember g component my-component-name

This is how the page looks like after I’m done.

Open app/templates/index.hbs.

<div class="row">
   <div class="col s12 m8 offset-m2">
        <h2 class="tooltipped" data-position="bottom" data-delay="50" data-tooltip="No, no one actually shouts, in case you thought that.">Shout feed</h2>
   </div>
</div>
{{post-input action='addPost'}}
{{#post-feed posts=model}}
   {{outlet}}
{{/post-feed}}

New Stuff!

  1. {{post-input action=’addPost’}} is a component in ember.js. action is like an HTML attribute, except that we can have any kind of user-defined attributes here.
    Important Note: Think of components as some kind of alternate HTML5 tags. If you notice, post-input looks like a self closing tag. #post-feed looks like a tag that has an additional closing tag, like most tags.  Components that need a closing tag are prefixed by a #. All components can be self-closing, or not. In my case, in case you haven’t figured out already, due to the absence of any sub-routes, the outlet is useless, so #post-feed might as well have been a self closing tag.
  2. Data Downs, Actions Up. This is a rule-of-hand, that you should pass data down into the layers of components, and retrieve actions generated by the components. In this case, the app/routes/index.js file is the file responsible for handling everything about app/templates/index.hbs, notice the same name? Then post-input component sends an action, the index route file will receive and act upon it, and when the index route file gets some data, it passes that into the post-feed component, where that data is passed into HTML tags and rendered. Both the components have their respective JS files for handing data and passing actions and etc.
  3. Tooltip. That is a Materialize thing. Pretty handy.

 

Generating components:

Components can be generated using:

ember g component your-component-name

Let’s look at the component templates after I’m done working on them:

app/templates/components/post-input.hbs

<div class="row">
    <form class="col s12 m8 offset-m2" {{action 'submitPost' on='submit'}}>
        <div class="row">
            <div class="input-field col s12">
            {{input id='new-post-title' type='text' value=postTitle}}
                <label for="new-post-title">Post Title</label>
         </div>
            <div class="input-field col s12">
            {{textarea id='new-post-body' class='materialize-textarea' value=postBody}}
                <label for="new-post-body">Post Body</label>
         </div>
            <div class="input-field col s12">
                <button class="btn">Shout<i class="mdi-content-send right"></i></button>
            </div>
        </div>
    </form>
</div>

New Stuff!

  1. input and textarea helpers. When using any kind of form fields in your app, the problem with using standard HTML tags would be that, ember won’t be able to bind itself to it. Using these helpers allows ember to implement something called data-binding. Now when the user enters values in then, these values are immediately reflected in this compoents JS file and in the local ember data-store.
  2. {{action ‘submitPost’ on=’submit’}}
    This means, when the form is submitted, the post-input (this component) will generate an action that will invoke the submitPost function in the app/components/post-input.js file. We’ll look into that soon.

app/templates/components/post-feed.hbs

{{#if posts}}
    <ul class="post-feed col s12 m8 offset-m2">
      {{#each posts as |post|}}
            <li class="post-item">
            {{post-item post=post}}
            </li>
      {{/each}}
    </ul>
{{else}}
    <div class="center" style="width:100%;">
        <h5>No cookie for you.</h5>
        <h6>No shouts available.</h6>
    </div>
{{/if}}

New Stuff!

  1. Conditionals. Statements like if and unless are implemented as built-in helpers. It’ll be great if you read about these in detail on ember guides.
  2. Iterators. Read about how to iterate over a list of items here and here.
  3. What’s happening here? You have seen in app/templates/index.hbs that we passed a model value to a posts attribute in the posts-feed tag. (Note: They aren’t officially called attributes or tags, I’m just using those terms so that you can relate better with HTML). We see that this model, that we’ll later find out contains a list of posts, is now being processed as the variable posts inside the template. the #each helper takes the list posts and iterates it through the variable post, which we see is subsequently passed into a smaller component named post-item. It should be obvious what it’s doing there.
  4. Note: posts is actually a JSON array. So post will be a JSON object in each iteration of that object array.

app/templates/components/post-item.hbs

<div class="col s12">
    <div class="card" style="min-height: 240px;">
        <div class="card-content">
            <span class="card-title grey-text text-darken-4 activator">{{post.title}}<i class="mdi-navigation-more-vert right waves-effect"></i></span>
            <p class="">{{post.body}}</p>
        </div>
        <div class="card-reveal">
            <span class="card-title grey-text text-darken-4">About {{post.userFullName}}<i class="mdi-navigation-close right waves-effect"></i></span>
            <p>{{post.aboutUser}}</p>
            <p>Email: <a href="mailto:{{post.newUser}}?subject=Connecting%20through%20ShoutOut&amp;body=Hey%2C%20I'm%20%5Benter-name%5D%2C%20I%20saw%20your%20shout%20on%20shoutout%20and%20found%20it%20interesting%2C%20how%20about%20we%20get%20in%20touch%3F" target="_blank">{{post.newUser}}</a></p>
            <p>Born on: {{post.userDOB}}</p>
            <p>Gender: {{post.gender}}</p>
        </div>
    </div>
    <span style="display: none;" class="post-time">{{post.postTime}}</span>
</div>

Since post was a JSON object, the variables in it are referenced the way usual JSON object keys are referenced.

 

We’ll now look at their respective JS files.

app/components/post-item.js

import Ember from 'ember';

export default Ember.Component.extend({
   classNames: ['row'],
   willRender: function () {
      console.log('Post rendered!')
   }
});

New Stuff!

  1. classNames: [‘row’], by default, your components are rendered inside a div tag generated by ember. You can add classes to that tag using the classNames property of a component.
  2. tagName: ‘li’, although not used here, even though there were multiple opportunities of using it, this property of a component does exactly what you might guess. By default, the tag that ember contains your component code in is a DIV tag, you can use this property to change that.
  3. willRender(), it is a component lifecycle hook that executes immediately before a component will be rendered. So in my case, if my app has 10 posts, the post-item template is getting rendered 10 times, and this function will be executed 10 times. There are many more lifecycle functions. Unfortunately, you’ll have to google that up.

app/components/post-feed.js

import Ember from 'ember';

export default Ember.Component.extend({
   postStatus: 'loading',
   classNames: ['row'],
   didRender: function () {
      let secondLastPostTime = $(".post-item:nth-last-child(2) .row .post-time").text();
      let lastPost = $(".post-item:last");
      let lastPostTime = $(".post-item:last .row .post-time").text();

      if(lastPostTime>secondLastPostTime){
         $(".post-item:last").remove();
         lastPost.css('display','none');
         $(".post-feed").prepend(lastPost);
         lastPost.slideDown(500);
      }
      $(document).ready(function(){
         $('.tooltipped').tooltip({delay: 50});
      });
   }
});

There’s some extra stuff here, that you don’t need to know right now, so focus on only that I’m going to mention:

New Stuff!

  1. didRender(), is it another ember component lifecycle hook that executes right after a component was rendered. In my case, the post feed will be rendered only when the page is loaded, or a new post is added.

app/components/post-input.js

import Ember from 'ember';

export default Ember.Component.extend({
   postTitle: '',
   postBody: '',
   actions: {
      submitPost(){
         let title = this.get('postTitle');
         let body = this.get('postBody');
         this.set('postTitle','');
         this.set('postBody','');
         if(title.length>0 && body.length>0){
            this.sendAction('action',title,body);
         }
         else{
            Materialize.toast('Both post title and body need to be entered.',3000);
         }
         //console.log(title)
      }
   }
});

Now this is where things start to get interesting.

New Stuff!

  1. Data-bindings, remember in it’s respective template files, we had postTitle bound to an input helper and postBody bound to a textarea helper? This is where it is accessible. On submitting the form, the action submitPost was executed. Inside this function, we can get() values that were bound to helpers in that template.
  2. sendAction(‘action’,title,body), scroll up and see that in the index.hbs template, you’ll see, it has an attribute named action and it invokes a function called addPost. Now we’ll see what it does and how it’s handled, and why whatever it does, wasn’t done right in this file.

app/routes/index.js 

import Ember from 'ember';

export default Ember.Route.extend({
   beforeModel: function () {
      let userdata = Cookies.getJSON('userdata');
      if(userdata){
         //load some data into the index template
      }
      else{
         //this.transitionTo('profile');
         //console.log('no user');
      }
   },
   model(){
      return this.store.query('post-item',{orderBy:'postTime', async: true});
   },
   actions: {
      addPost(title,body){
         let userdata = Cookies.getJSON('userdata');
         this.store.createRecord('post-item',{
            title: title,
            body: body,

            userFullName: userdata.userFullName,
            gender: userdata.gender,
            newUser: userdata.newUser,
            aboutUser: userdata.aboutUser,
            userDOB: userdata.userDOB
         }).save();
         console.log(title)
      }
   }
});

New Stuff!

  1. Route file, this file is the main file that handles all the functionality under a certain route with the same path name, unless a seperate pathname is specified in the router.js.
  2. beforeModel(), this route lifecycle hook executes before the data for this route is fetched by the model() function.
  3. model(), this function gets the data into a model associated with the app. We’ll discuss models soon.
  4. actions{}, you have seen this in multiple files by now. actions{}, specifies the functions this router can perform. In index.hbs, the post-input component sent an action via the sendAction() method with two parameters, that action was received by the router, and transferred to the addPost(), method with the same parameters.
  5. transitionTo(), in the beforeModel(), transitionTo() has been commented out here for tutorial reasons, but you should know, in case you want to change the route programmatically instead of having the use click on a link-to helper, you can use transitionTo(‘route-name’) function as specified.

 

Section D – Learning about models

You can learn all you need about ember.js models here.

Here are some of the important parts.

Models are objects that represent the underlying data that your application presents to the user. Different apps will have very different models, depending on what problems they’re trying to solve.

For example, a photo sharing application might have a Photo model to represent a particular photo, and a PhotoAlbum that represents a group of photos. In contrast, an online shopping app would probably have different models, like ShoppingCart, Invoice, or LineItem.

Models tend to be persistent. That means the user does not expect model data to be lost when they close their browser window. To make sure no data is lost, if the user makes changes to a model, you need to store the model data somewhere that it will not be lost.

Typically, most models are loaded from and saved to a server that uses a database to store data. Usually you will send JSON representations of models back and forth to an HTTP server that you have written. However, Ember makes it easy to use other durable storage, such as saving to the user’s hard disk with IndexedDB, or hosted storage solutions that let you avoid writing and hosting your own servers.

Ember Data, included by default when you create a new application, is a library that integrates tightly with Ember to make it easy to retrieve models from your server as JSON, save updates back to the server, and create new models in the browser.

Thanks to its use of the adapter pattern, Ember Data can be configured to work with many different kinds of backends. There is an entire ecosystem of adapters that allow your Ember app to talk to different types of servers without you writing any networking code.

This is how you use ember-cli to generate a model:

ember g model my-model key1:string key2:number key3:date key4:one-of-the-4-data-types

This is how our model looks like:

import DS from 'ember-data';

export default DS.Model.extend({
   title: DS.attr('string'),
   body: DS.attr('string'),
   userFullName: DS.attr('string'),
   gender: DS.attr('string'),
   newUser: DS.attr('string'),
   aboutUser: DS.attr('string'),
   userDOB: DS.attr('string'),
   postTime: DS.attr('number',{
      defaultValue: function () {
         return new Date().getTime()*(-1);
      }
   })
});

It’s not hard to guess how this may have been generated, however, postTime needs to be hand-typed anyway.

Part IV – Getting it over with

By now, your app is pretty much ready. However, if you never used Firebase, you probably got stuck with it earlier, and here we are, back to it.

Let’s get started!

  1. Go to https://www.firebase.com/signup/ and sign up.
  2. Once logged in, create your new app. If you name your add something like myfirstemberapp, your firebase URL would be myfirstemberapp.firebaseio.com, and that’s why choosing project names or URLs here would be like choosing usernames.
  3. Get the URL you just created and paste it where you were asked you in Part I.
  4. That is literally it.

Now you have the ability to share posts in real-time with other users and sync it in a firebase DB all with just this much effort.

Part V – Conclusion

So, did your app work? If not, what went wrong? If yes, congratulations! Did you have any difficulty? Where so? How do you think I can improve this tutorial? I will improve and expand it further anyway actually. Till then, feedback is welcome. 🙂

A glimpse of undefined behavior in C

Reblogged from “A glimpse of undefined behavior in C” at Christopher Cole’s blog, “Christopher Cole” .

Hey! Today I was practicing my first year CS201 programming language, C, when I came across a puzzling situation. Several lines of the following code gave different output on different compilers. Which also differed from what the output should logically be.

#include
int main()
{
        int i=5;
        printf("%d %d %d\n",i,++i,i++); //7 7 5
        i=5;
        printf("%d %d %d\n",i,i++,++i); //7 6 7
        i=5;
        printf("%d %d %d\n",i,++i,++i); //7 7 7
        i=5;
        printf("%d %d %d\n",i,i++,i++); //7 6 5
        i=5;
        printf("%d %d %d\n",i++,i++,i); //6 5 7
        i=1;
        printf("%d %d %d\n",i++,i++,i); //2 1 3
        return 0;
}

And so I started googling the possible reasons for that happening. Several stack exchange pages told me that this is an “Undefined Behaviour” that happens in C and C++. But why would it be undefined? What really goes on behind the scenes? Asking my friends for help on facebook led me to the aforementioned blog post that breaks the program down to assembly language which helped me understand what was really going on. I’ve reblogged that post here. Hope it helps. Also, I should mention that this didn’t help me understand WHY it was happening. I merely assumed that it was happening because of the way the compiler processes the code.

Here is the reblogged post.

A few weeks ago a coworker of mine stopped by my desk with a coding question. We’d recently been quizzing each other over our C knowledge, so I smiled and braced myself for the hell which was clearly to come.

He wrote a few lines of code up on the whiteboard and asked what the program output:

#include
int main(){
int i = 0;
int a[] = {10,20,30};

int r = 1 * a[i++] + 2 * a[i++] + 3 * a[i++];
printf("%d\n", r);
return 0;
}

It seemed pretty straightforward. I explained the order of operator precedence – postfix binds before multiplication which binds before addition and also that multiplication and addition’s associativity was left to right, so I grabbed the marker and started writing out the arithmetic.

int r = 1 * a[i++] + 2 * a[i++] + 3 * a[i++];
// = a[0] + 2 * a[1] + 3 * a[2];
// = 10 + 40 + 90;
// = 140

After I smugly scribbled down the answer, my coworker replied a simple “nope”. I thought for a few minutes, and was stumped. I couldn’t quite remember the order of the postfix operators’ associativity. Furthermore, I knew that wouldn’t even change the order of evaluation here as associativity rules only apply between operators of the same precedence level, but thought I’d give it a shot and walk through the arithmetic as if the postfix operators were all evaluated right to left.

int r = 1 * a[i++] + 2 * a[i++] + 3 * a[i++];
// = a[2] + 2 * a[1] + 3 * a[0];
// = 30 + 40 + 30;
// = 100

Again, my coworker replied it was still wrong. At this point I threw up my hands and asked what the answer was. It turned out that this small sample was actually reduced from a larger code snippet he’d put together. To verify his question, he’d compiled and run the larger sample but was surprised to see it not behave as he’d thought. After reducing the unexpected behavior to this illuminating example, he’d compiled it with gcc 4.7.3 and it output a startling result: “60″.

At this point I was intrigued. I remembered that, in C, the order of function argument evaluation was undefined, so we thought maybe the postfix operators were just being evaluated in some random non-left-to-right order. Still convinced that postfix had a higher operator precedence than addition and multiplication, we quickly proved to ourselves that there was no order we could evaluate the i++’s such that the three elements of the array were added/multiplied together to get “60″.

Now I was hooked. My first thought was to look at the disassembly of the snippet and try and track down what was actually going on. I compiled the sample with debugging symbols and after a quick objdump we had the source-annotated x86_64 disassembly:

Disassembly of section .text:

0000000000000000 :
#include

int main(){
0: 55 push %rbp
1: 48 89 e5 mov %rsp,%rbp
4: 48 83 ec 20 sub $0x20,%rsp
int i = 0;
8: c7 45 e8 00 00 00 00 movl $0x0,-0x18(%rbp)
int a[] = {10,20,30};
f: c7 45 f0 0a 00 00 00 movl $0xa,-0x10(%rbp)
16: c7 45 f4 14 00 00 00 movl $0x14,-0xc(%rbp)
1d: c7 45 f8 1e 00 00 00 movl $0x1e,-0x8(%rbp)
int r = 1 * a[i++] + 2 * a[i++] + 3 * a[i++];
24: 8b 45 e8 mov -0x18(%rbp),%eax
27: 48 98 cltq
29: 8b 54 85 f0 mov -0x10(%rbp,%rax,4),%edx
2d: 8b 45 e8 mov -0x18(%rbp),%eax
30: 48 98 cltq
32: 8b 44 85 f0 mov -0x10(%rbp,%rax,4),%eax
36: 01 c0 add %eax,%eax
38: 8d 0c 02 lea (%rdx,%rax,1),%ecx
3b: 8b 45 e8 mov -0x18(%rbp),%eax
3e: 48 98 cltq
40: 8b 54 85 f0 mov -0x10(%rbp,%rax,4),%edx
44: 89 d0 mov %edx,%eax
46: 01 c0 add %eax,%eax
48: 01 d0 add %edx,%eax
4a: 01 c8 add %ecx,%eax
4c: 89 45 ec mov %eax,-0x14(%rbp)
4f: 83 45 e8 01 addl $0x1,-0x18(%rbp)
53: 83 45 e8 01 addl $0x1,-0x18(%rbp)
57: 83 45 e8 01 addl $0x1,-0x18(%rbp)
printf("%d\n", r);
5b: 8b 45 ec mov -0x14(%rbp),%eax
5e: 89 c6 mov %eax,%esi
60: bf 00 00 00 00 mov $0x0,%edi
65: b8 00 00 00 00 mov $0x0,%eax
6a: e8 00 00 00 00 callq 6f 
return 0;
6f: b8 00 00 00 00 mov $0x0,%eax
}
74: c9 leaveq
75: c3 retq

The first few and last few instructions just set up the stack frame, initial variable values, and do the printf call and return from main, so we really only cared about instructions 0×24 through 0×57. That’s where the interesting behavior was. Let’s walk through the instructions a few at a time:

24: 8b 45 e8 mov -0x18(%rbp),%eax
27: 48 98 cltq
29: 8b 54 85 f0 mov -0x10(%rbp,%rax,4),%edx

The first three instructions were as we expected. First, it loads the value of i (0) into register eax, sign-extends it to 64 bits, then loads a[0] into register edx. The 1 * was obviously optimized out by the compiler, but everything looked normal here. The next few started out much the same:

2d: 8b 45 e8 mov -0x18(%rbp),%eax
30: 48 98 cltq
32: 8b 44 85 f0 mov -0x10(%rbp,%rax,4),%eax
36: 01 c0 add %eax,%eax
38: 8d 0c 02 lea (%rdx,%rax,1),%ecx

The first mov loads the value of i (still 0) into eax, sign-extends it to 64-bits, then loads a[0] into eax. That’s intersting – we expected the i++ to have been performed before these three instructions again, but maybe the last two instructions will do some sort of assembly magic to get us to the expected result (2 * a[1]). The two instructions add eax to itself, effectively performing 2 * a[0], then add that result to the previous computation and stores it in ecx. At this point the instructions have evaluated a[0] + 2 * a[0]. Things are starting to look a little strange but again, maybe there’s some compiler magic going on.

3b: 8b 45 e8 mov -0x18(%rbp),%eax
3e: 48 98 cltq
40: 8b 54 85 f0 mov -0x10(%rbp,%rax,4),%edx
44: 89 d0 mov %edx,%eax

These next few instructions are starting to look pretty familiar. They load the value of i (still 0), sign extend it to 64-bits, load a[0] into edx, then copy edx into eax. Hmm, ok let’s take a look at a few more:

46: 01 c0 add %eax,%eax
48: 01 d0 add %edx,%eax
4a: 01 c8 add %ecx,%eax
4c: 89 45 ec mov %eax,-0x14(%rbp)

Here, it adds a[0] to itself 3 times, adds it the previous computation, then stores it into variable ‘r’. Now that’s wild – our variable r now contains a[0] + 2 * a[0] + 3 * a[0]. Sure enough, that’s what the program output: “60″. But what happened to those postfix operations? They’re all there at the end:

4f: 83 45 e8 01 addl $0x1,-0x18(%rbp)
53: 83 45 e8 01 addl $0x1,-0x18(%rbp)
57: 83 45 e8 01 addl $0x1,-0x18(%rbp)

It seemed our compiled version of the code just seemed completely wrong! Why had the postfix operations been thrown down at the end after the assignment had already happened? With my faith in reality dwindling, I decided to go straight to the source. No, not the compiler’s source – that’s just an implementation – I grabbed the C11 language specification.

The issue lay within the details of the postfix operations. In our case, we were thrice performing a postfix increment to our array index in a single expression. When evaluating a postfix operator it returns the initial value of the variable. The assignment of the new value back to the variable is known as a side effect. It turns out that side effects are only defined to have been committed between sequence points. See section 5.1.2.3 of the standard for the specifics of where sequence points are defined, but in our case our expression exhibits undefined behavior. It’s entirely up to the compiler when the side-effect of the assignment of the new value back to the variable is performed with respect to the other parts of the expression.

In the end, we both learned a bit more about the C language. It’s well-known best practice to avoid constructing complex expressions with prefix/postfix operators, and this is a great example of why that is the case.

The sample program and the output on my computer.
The sample program and the output on my computer.
%d bloggers like this: