Blog and Labs

Last Monday was a bank holiday here in the UK, and while most people were playing computer games or out socialising, our CTO, Richard, was starting to truly think with portals. After releasing a ruby wrapper for the Pachube TCP stream API, Richard decided to build a Node.js streaming TCP client for Pachube's TCP server.

Node.js is an event-driven I/O framework for server-side JavaScript, but what does this mean and what does it allow us to do? Well, it essentially allows us to open up portals between events, allowing us to listen for information, rather than asking for it via the web:

Combining this with the Pachube TCP stream API enables us to extend this to real-world devices, creating portals for data to stream through to our software. What are we doing with these portals and this data?...You'll just have to wait and see ;-)

We have of course released the code for this client via the Carbon Calculated Github account; feel free to fork, commit or comment.

We have just released a gem (Ruby plugin) which uses the new beta Pachube TCP stream using Eventmachine.

The Gem is available on the Carbon Calculated github here: https://github.com/carboncalculated/pachube-stream

Pachube allows you to manage and sensor realtime data such as electricity, and have recently partnered with CurrentCost acting as the data store for the CurrentCost web bridge.

Our gem allows a Ruby developer to access their own or publically available data streams from pachube and consume the data inside their own applications. Here at CarbonCalculated we are using the Pachube Stream gem to enable realtime monitoring of electricity, gas and water consumption and related carbon emissions across our range of products. This means there is no longer a need to import gas, electricty or water bills, simply fit a web enabled meter and add the readings from the device straight into your companies carbon footprint!

To get started with the gem, read the documentation on github and have a look at the examples here: https://github.com/carboncalculated/pachube-stream/tree/master/examples

The pachube gem is open source and therefore pull requests are very much welcome!

We at Carbon Calculated always look at the advances in technology and how we can use them for developing new application for carbon calculation.

We have been experimenting with Sproutcore (Javascript MVC framework) recently and have knocked up a quick example application:

Electricity Carbon Calculator example app

and the code for this is open source and availble here:

https://github.com/carboncalculated/electric-bill-carbon-calculator-sproutcore

This shows how quick and easy Sproutcore and the Carbon Calculated API are to use.

For those how have been enquiring about our admin system for data imports and managing the data in the platform, I have attached a screenshoot of the admin system. Via setting data to specific requirements we have the facility to import vast amounts of data, and with this rich interface we can easily manage any data requests that we have. We will in the future be allowing other parties to access the admin so they can update their specific data.

We have just release first in a series of open source applications to help developers get a better understanding of the carbon calculated platform.

Rails3

The application is in Rails3 and uses the calculated gem

Application

You can find the application here

Gem

You can find the gem here

Firstly as per normal you will need an API KEY so contact support@carboncalculated.com

Install

View the source HERE

sudo gem install calculated

Finding Emission of Cars

To give a very small example of calculating car emissions lets find a calculator for “cars”

• Lets go to the browser and see what car calculators there are HERE

• We then inspect the “Car Emissions by distance” computation by distance

• From here we can determine what we need to request to gain a valid answer

• We first need to get a CAR so lets try and GET A CAR

Getting A Session

@session = Calculated::Session.create(:api_key => "your_api_key")

Getting A Car

I want a ford car that was registered in 2008; ok so how do I do that? I use the browser I find the relatable categories which are FORD and 2008

@cars = @session.related_objects_from_relatable_categories("car", ["4c3713f7dc82a16f44003f4b", "4c370ad56f67596a98002091"])

Now we have a list of cars so lets choose one I am going for “Fiesta, 2008 Model Year Onwards – 1.6 Duratec (100PS) – 1596cc – A4 – Petrol – May onwards” ID “4c37149594343e40bf00558b”

Getting An Answer

From the required parameters that we found earlier; we can now make a valid request for a result.

@result = @session.answer_for_computation("4c2b11f2ba5e45da86000002", {:car => "4c37149594343e40bf00558b", :formula_input_name => "emissions_in_kg_per_km",  :distance => 10})
@result.calculations["co2"]["value"] # 1.76

Got Any questions???

And Look if we travel not 10Km But 100km we get a different result

@result = @session.answer_for_computation("4c2b11f2ba5e45da86000002", {:car => "4c37149594343e40bf00558b", :formula_input_name => "emissions_in_kg_per_km",  :distance => 100})
@result.calculations["co2"]["value"] # 17.6

How Do I Get A List of Manufacturer Names??

@object_template = @session.relatable_categories_for_object_template("car", "manufacturer")
@object_template.relatable_categories.map(&:name)

Remember

That this will only bring back 50 Relatable Categories therefore you will need to know the full amount of manufacturers if you wish to have the whole list; this again can be found via the browser

@object_template = @session.relatable_categories_for_object_template("car", "manufacturer", :per_page => 60)

Just thought I would give a quick outline of the features that we released form day 1 of the platforms release; and also the power of having validations at the CORE of our calculators; and what this means to developers.

Greenhouse gases

From the first release of the API we have supported other Greenhouse gases (CH4, N20, CO2E, indirect CO2E, direct CO2E, direct CO2, indirect CO2) besides CO2.

This can be shown here on the browser

This rich calculation information can really help if you need to build an application for the CDP (carbon disclosure project) for instance

Information When Calculating

When you ask for a result from the platform it well tell you what is used in its calculation; This enables the developer to use these results for all kinds of possibilities. To Show a quick example of this lets get an emission result for traveling by plane from Heathrow to JFK

Path

/calculators/4bab7e17f78b122cdd000001/answer.json?api_key=your_api_key&transport_type=4bd5cb38dfde7b12560033db&transport=4bd5cb39dfde7b12560033ff&no_of_journeys=1&no_of_people=1&formula_input_name=per_passenger_km&origin=4b83f9af0e61b2220d00538d&destination=4b83f9db0e61b2220d005faa

If you do not understand this request please look at the API documentation; however going to the browser may help to understand the parameters that will be sent

This request returns the following result

{"calculations":{"co2":{"value":681.049338,"units":"kg"},"n2o":{"value":6.676954,"units":"kg CO2e"},"ch4":{"value":0.0607,"units":"kg CO2e"},"co2e":{"value":687.786992,"units":"kg"}},"answer_set_id":"4bd5cb3bdfde7b125600377f","source":{"id":"4ca48e57fd516d1b5b000014","description":"","main_source_ids":["4bebea80dfde7b477d000008"],"external_url":"http://miniapps.carboncalculated.com/journey/calculation/answers/new","wave_id":"me9YDh_mC"},"calculator_id":"4bab7e17f78b122cdd000001","computation_id":"4bcc683e1076f1533f000001","object_references":{"4b83f9af0e61b2220d00538d":{"id":"4b83f9af0e61b2220d00538d","template_name":"airport","characteristics":[{"attribute":"label","value":"United Kingdom, London - LGW","units":null,"value_type":"String","id":null},{"attribute":"airport_name","value":"London - LGW","units":null,"value_type":"String","id":null},{"attribute":"latitude","value":"51.15","units":null,"value_type":"Float","id":null},{"attribute":"longitude","value":"-0.19","units":null,"value_type":"Float","id":null},{"attribute":"iata_code","value":"LGW","units":null,"value_type":"String","id":null},{"attribute":"airport","value":"Gatwick","units":null,"value_type":"String","id":null},{"attribute":"country","value":"United Kingdom","units":null,"value_type":"String","id":null}],"formula_inputs":[],"identifier":"United Kingdom, London - LGW - Gatwick"},"4b83f9db0e61b2220d005faa":{"id":"4b83f9db0e61b2220d005faa","template_name":"airport","characteristics":[{"attribute":"label","value":"United States, New York - JFK","units":null,"value_type":"String","id":null},{"attribute":"airport_name","value":"New York - JFK","units":null,"value_type":"String","id":null},{"attribute":"latitude","value":"40.64","units":null,"value_type":"Float","id":null},{"attribute":"longitude","value":"-73.78","units":null,"value_type":"Float","id":null},{"attribute":"iata_code","value":"JFK","units":null,"value_type":"String","id":null},{"attribute":"airport","value":"John F Kennedy International","units":null,"value_type":"String","id":null},{"attribute":"country","value":"United States","units":null,"value_type":"String","id":null}],"formula_inputs":[],"identifier":"United States, New York - JFK - John F Kennedy International"},"4bd5cb39dfde7b12560033ff":{"id":"4bd5cb39dfde7b12560033ff","template_name":"transport","characteristics":[{"attribute":"label","value":"Long Haul International - Average","units":null,"value_type":"String","id":"4bd5cb39dfde7b1256003400"},{"attribute":"transport_type","value":"plane","units":null,"value_type":"String","id":"4bd5cb39dfde7b1256003401"},{"attribute":"flight_length","value":"Long-haul international","units":null,"value_type":"String","id":"4bd5cb39dfde7b1256003404"},{"attribute":"travel_class","value":"Average","units":null,"value_type":"String","id":"4bd5cb39dfde7b1256003405"}],"formula_inputs":[{"model_state":"active","name":"per_passenger_km","group_name":null,"values":{"km_uplift_factor":1.09,"co2":0.1122,"ch4":1.0e-05,"n2o":0.0011,"co2e":0.11331},"active_at":"2010-04-26T17:19:50Z","input_units":"km","label_input_units":"km per passenger","main_source_id":null,"source":null,"id":null}],"identifier":"Long Haul International - Average","used_formula_inputs":{"per_passenger_km":{"model_state":"active","name":"per_passenger_km","group_name":null,"values":{"km_uplift_factor":1.09,"co2":0.1122,"ch4":1.0e-05,"n2o":0.0011,"co2e":0.11331},"active_at":"2010-04-26T17:19:50Z","input_units":"km","label_input_units":"km per passenger","main_source_id":null,"source":null,"id":null}}}},"used_global_computations":{"haversine_distance_degrees":5568.769219237845}

This has vast of information for the developers so let me highlight a few key aspects of what we get back;

Calculations

{"calculations":{"co2":{"value":681.049338,"units":"kg"},"n2o":{"value":6.676954,"units":"kg CO2e"},"ch4":{"value":0.0607,"units":"kg CO2e"},"co2e":{"value":687.786992,"units":"kg"}}

Main Source Ids

4bebea80dfde7b477d000008"

This allow you to link to the browser to show your user where and how the result was obtained

http://browser.carboncalculated.com/main_sources/4bebea80dfde7b477d000008

Object References

{"id":"4b83f9af0e61b2220d00538d","template_name":"airport","characteristics":[{"attribute":"label","value":"United Kingdom, London - LGW","units":null,"value_type":"String","id":null},{"attribute":"airport_name","value":"London - LGW","units":null,"value_type":"String","id":null},{"attribute":"latitude","value":"51.15","units":null,"value_type":"Float","id":null},{"attribute":"longitude","value":"-0.19","units":null,"value_type":"Float","id":null},{"attribute":"iata_code","value":"LGW","units":null,"value_type":"String","id":null},{"attribute":"airport","value":"Gatwick","units":null,"value_type":"String","id":null},{"attribute":"country","value":"United Kingdom","units":null,"value_type":"String","id":null}],"formula_inputs":[],"identifier":"United Kingdom, London - LGW - Gatwick"}

All the generic objects (objects) that were used in the calculation will be returned; this enables a developer to have access to characteristics that these objects have and utilise them accordingly;

The airport JFK will bring back its latitude, longitude, and country;

The transport object “long haul flight” will tell you its uplight factor its attached emissions values and where they came from; via its source id; this can then be used to inform the end user exactly how the result was calculated;

Used Computations

"used_global_computations":{"haversine_distance_degrees":5568.769219237845}

Here we can clearly see that we get the distance from the two airports; it even explains the formula used to get that distance; how transparent is that!

Super Calculators

Having validations at the CORE of the calculators enables us to give developers powerful features; we use these features here to build automatic UIs for our calculators that we build; Our QA application are an example of this;

Lets find the emissions of travelling by London Bus for 10Km

/calculators/4bab7e17f78b122cdd000001/answer.json?api_key=your_api_key&transport_type=4bd5cb38dfde7b1256003360&transport=4bd5cb38dfde7b1256003361&no_of_journeys=1&no_of_people=1&distance=10&formula_input_name=per_passenger_km

Look!!!!!!!!!!

/calculators/4bab7e17f78b122cdd000001/

this the same calculator for the previous flight calculation we did; But this calculator contains many computations; one for flight one for flight using airport; and one for other transport vehicles.

How can this work??? How does it know what to calculated? Validations

Why is this so cool?

You the developer may not know what the user wants to calculate; however you know it evolves transport!

Therefore you can just use the passenger transport calculator to determine what kind of transport calculation it need to use

Now when you create a Journey calculator for instance; you dont have to create logic in determining what calculation to use; just ask the platform to determine it for you;

Its hard to convey how powerful this feature is? But if you into carbon calculations; please look at our new QA application for Defra Waste emissions; this has 16 separate computations and yet it we only have to send the users answers to to the 1 calculator; this means no extra logic; no extra code; time saved; LOTS!

Today we are going to be build a simple journey application using the carbon calculated API;

Something a bit like

Steps

1. We know the question! but how do we get an answer; simple! lets go to the browser and select the correct calculator in this case the passenger transport calculator

When we look at this calculator we can see that there are 3 computations: Passenger Flight via IATA code; Passenger Transport(excluding flight); And Passenger Flight (By Distance)

In the Carbon Calculated API you can get an answer from either a computation directly or the calculator! The power of this should be visible from the screenshoot; The user can decide to choose from a flight mode of transport or something else; at run time, and then we can use just send those parameters to the API without any other processing; we can let the API decide on its computation.

This then allows you the developer to ask the question in a very simple way; Hopefully you see what I mean as we move through the application.

From a previous post we have already indicated how you can create successive drop downs; Therefore we can see that the drop downs were generate from transport Objects using its relatable categories; This then gives the user sensible filtering options;

But here is how the transport type drop is populated;

/object_templates/transport/relatable_categories.json?api_key=your_api_key&related_attribute=transport_type

Then the Vehicle drop down

/relatable_categories/related_objects.json?api_key=your_api_key&relatable_category_ids[]=4bd5cb38dfde7b1256003393&template_name=transport

So What about when we select Flights

From below we can see that it has generated 2 airport searches; when the user has selected both airports the computation invoking airport iata codes can be used shown here

The page can submit to the same calculator regardless of whether the user selected the aiports or not because the api can determine from its validations which computation to use;

For those that are curious a search on objects is performed like below; We are searching for an airport in london!

/object_templates/airport/generic_objects/filter.json?api_key=your_api_key&filter=london&relatable_category_values=&template_name=airport

So for the finale; we need to get an answer; however as application developers we only need to send a GET request in a specific way; we can let the users decide the rest of the parameters at runtime; allowing for flexible easy to build interfaces;

User selects and Enters Airport Information

GET REQUEST

/calculators/4bab7e17f78b122cdd000001/answer.json?api_key=your_api_key&calculator_id=4bab7e17f78b122cdd000001&destination=4b83f9db0e61b2220d005faa&formula_input_name=per_passenger_km&no_of_journeys=1&no_of_people=1&origin=4b83f9af0e61b2220d00538d&transport=4bd5cb38dfde7b12560033de&transport_type=4bd5cb38dfde7b12560033db

User selected a bus and enters the rest of the information

GET REQUEST

/calculators/4bab7e17f78b122cdd000001/answer.json?api_key=your_api_key&calculator_id=4bab7e17f78b122cdd000001&distance=15&formula_input_name=per_passenger_km&no_of_journeys=1&no_of_people=1&transport=4bd5cb38dfde7b1256003361&transport_type=4bd5cb38dfde7b1256003360

Any questions just comment away!

When you come to use mongoDB for the first time, it may seem strange. It may even be alien if you come from a relational database background- “don’t we all”.

What you will find very quickly though, is that relational joins are the most over used model design in the world. I have almost forgotten the days when I would instantly think: “thats a join”, “oh thats a join” and “well thats a join” and “well that looks like join”.

We can do soooo much better than that in our model design!

I say, the less the “normalization” the better.

I could simply write about all the features of MongoDB and why they are cool, but thats whats the mongoDB website is for.

Schemaless In Design

Carbon Calculated use MongoMapper to help us in our MongoDB foo; and clicking on that link will treat you to a better explanation than I can do.

There are many aspects that I could try to convey but I will keep this post short and sweet:

1. When an Array is an Array; It be stored as an Array
2. When a Hash is a Hash; It be stored as a Hash
3. Why do I have to “join” when I can embed!

If that is not enough for you, then Relational you will always be!

For those that do not know; some of the tools we use for carbon calculation domination are:

Ruby

“A dynamic, open source programming language with a focus on simplicity and productivity. It has an elegant syntax that is natural to read and easy to write.”

Rack

“Rack provides a minimal, modular and adaptable interface for developing web applications in Ruby. By wrapping HTTP requests and responses in the simplest way possible, it unifies and distills the API for webservers, web frameworks, and software in between (the so-called middleware) into a single method call.”

How we use Rack

We used rack to help us modularise separate aspects of the API. The answer section in the API is therefore a completely separate application, compared, say, to the generic_objects application. This means that we can scale each separate application accordingly or combine each separate application to make one; which is what we serve today.

Another huge benefit of this structure is that we can version our API incredible easily; scale easily and update individual aspects of the API without affecting other aspects.

module CarbonCalculatedApi
    module API
      def self.app

        # == Build Version 1 Application
        @version1 ||= Rack::Builder.app do
          use CarbonCalculatedApi::GenericObjectApp
          use CarbonCalculatedApi::AnswerApp
          use CarbonCalculatedApi::ObjectTemplateApp
          run CarbonCalculatedApi::RelatableCategoryApp
        end

        @api_mapper ||= Rack::APIVersionMapper.new do |m|
          m.add_version 1, @version1
          m.current_version = 1
        end
      end
    end
  end

I will also talk about how we use MongoDB but that is for another day