Strangeparty

Message Broker v8: Record, Replay and REST

Anton Piatek — Fri, 06 Jan 2012 09:06:59 +0000

IBM WebSphere Message Broker v8 was released in December last year – There is an announcement and comparison of versions on the IBM website.

IBM WebSphere Message Broker introduced Record and Replay as part of its version 8 release. I thought I would just write a brief quickstart guide to getting running with record and replay, and in particular with working with the new REST interface which comes with it. There is actually a Record and Replay specific PDF available too, which contains all the details you might want for using Record and Replay (far more detail than I cover here). The main reason I am writing this, is it can be a little daunting trying to get Record and Replay going, as it does have a fair bit of configuration, and involves a few areas of a quite large product.

Setting up Recording

From your WMB install dir, run the db2 schema file to create the sample database for Record and Replay. Edit the file if you want to use a database name other than MB8RECORD, a specific schema or increase the storage size per message above 5mb.

db2 -tvf ddl/db2/DataCaptureSchema.sql

Set the password for your database

mqsisetdbparms MB8BROKER -n MBRECORD -u username -p password

Set the DataCaptureStore configurable service to set where to record to and which EG to do the recording (You can also do any of these configurable service steps via the Message Broker Explorer gui)

mqsicreateconfigurableservice MB8BROKER -c DataCaptureStore -o MyCaptureStore \
   -n "dataSourceName,egForRecord" -v "MBRECORD,default"

Define where to record messages from. Use a wildcard to set all configurable services. Note that double quotes (“”) are required on unix, windows will want single quotes (”)

mqsicreateconfigurableservice MB8BROKER -c DataCaptureSource -o MyCaptureSource \
   -n "dataCaptureStore,topic" -v "MyCaptureStore,$SYS/Broker/MB8BROKER/Monitoring/#"

Alternatively you can list the flows explicitly with topics like “$SYS/Broker/MB8BROKER/Monitoring/ExecutionGroupName/FlowName” While we are configuring everything, set a replay destination too. Set EG default to replay to queue “REPLAY” on queue manager MB8QMGR

mqsicreateconfigurableservice MB8BROKER -c DataDestination -o MyDestination \
   -n "egForReplay,endpoint,endpointType" -v "default,wmq:/msg/queue/REPLAY@MB8QMGR,WMQDestination"

Setting up Monitoring

Now your broker is configured for recording, you will need to setup something to be recorded. Event Monitoring is used to emit data to record, so either follow the monitoring sample, read the docs on Event Monitoring or edit the sample monitoring profile in sample/RecordReplay/basicMonitoringProfile.xml to replace NODENAME with the name of your flow’s input node. Of course you can also configure event monitoring from within the flow editor in the toolkit – simply select a node and you will find monitoring in the properties view. If you have the flow source already, it is much easier to set the monitoring in the toolkit than it is to do it via the command line. You can also then use the mqsireportflowmonitoring command afterwards to give you the xml monitoring profile being used (which is basically what you are setting in the toolkit when setting the monitoring options on a flow)

There are two specific properties which are worth mentioning when it comes to monitoring for Record and Replay.
If you tick the include bitstream option in the toolkit then the body of your message will be recorded. Choose a sensible encoding, as if you select CDATA and your body will not be a valid XML element, you will break the monitoring and no event will be emitted. If in doubt, use the hexbinary or base64 encoding. It is worth saying that if you record MQ headers, and replay that message, the headers will be attempted to be reused. Other transport headers will just end up in the body of your message.
The other option I wanted to mention was the UserData area of Event Monitoring. If you include $ExceptionList in your monitoring settings, then if an error occurs and an event is emitted, the exception list will be included in the monitoring message. This will be recorded, and will be available from the Record and Replay web interface and REST Api.

If you used a monitoring profile you will need to configure the profile with the following commands:

mqsicreateconfigurableservice MB8BROKER -c MonitoringProfiles -o MyProfile

mqsichangeconfigurableservice MB8BROKER -c MonitoringProfiles -o MyProfile \
   -n profileProperties -p "sample/RecordReplay/basicMonitoringProfile.xml"

mqsichangeflowmonitoring MB8BROKER -e default -f MyFlow -m  MyProfile

Monitoring normally defaults to off, so you will probably need to turn it on with:

mqsichangeflowmonitoring MB8BROKER -e default -f MyFlow -c active

You can check that monitoring is correctly active with the command

mqsireportflowmonitoring MB8BROKER -n -e default -f MyFlow

You should see output like

BIP8911I: Monitoring settings for flow 'MyFlow' in execution group 'default' -  State?: active, ProfileName: 'MyProfile'.
BIP8912I: Event: 'IN.transaction.End',  Event name: 'IN.transaction.End',  Configured?: yes,  State?: enabled.
BIP8912I: Event: 'IN.transaction.Rollback',  Event name: 'IN.transaction.Rollback',  Configured?: yes,  State?: enabled.

You can now put messages through your flow and they will be recorded.

Enabling the REST interface and Web UI

To enable the REST interface and Web UI you need to enable the HTTP server and set the port for it to run on. You will also need to set which ExecutionGroup will process requests to view recorded messages. (You can also do this via the MBX properties, see the properties for your broker, and for your DataCaptureStore ConfigurableService)

mqsichangeproperties MB8BROKER -b webadmin -o server -n enabled -v true
mqsichangeproperties MB8BROKER -b webadmin -o HTTPConnector -n port -v 8080
mqsichangeproperties MB8BROKER -o DataCaptureStore -c MyCaptureStore -n egForView -v default

You can also configure HTTPS if you prefer, details are in the product documentation.

Viewing recorded data

You can of course view all the data via a web browser by pointing to the url http://localhost:8080 but you can also do it all via the command line. The following will get back the most recent 30 recorded pieces of data from your captureStore MyCaptureStore.

$curl "http://localhost:8080/datacapturestore/MyCaptureStore/"

This returns quite a large XML list of messages, so let’s try getting just one message.

$curl "http://localhost:8080/datacapturestore/DefaultCaptureStore/?numberOfEntriesPerPage=1"

You can of course filter on any of the data fields above as follows.

$curl "http://localhost:8080/datacapturestore/DefaultCaptureStore/?numberOfEntriesPerPage=1&has_bitstream=Y"

Filtering is possible on all the above columns, with the addition that you can also set a timestamp range with The REST Api has documentation at docs/REST/index.html which covers all these options.

You can also query which DataCaptureStores are setup for viewing by loading the base url “http://localhost:8080/datacapturestore/”

You may then want to download the body of a recorded message. (My example has a shortened url as the message key rather long. Use the wmb_msgkey from your above list requests) Depending on how you set the data to be recorded in your monitoring profile, you will get back a different encoding data

$curl "http://localhost:8080/datacapturestore/DefaultCaptureStore/12345"
SGVsbG8gV29ybGQK

You don’t have to get the data back as encoded payloads wrapped in XML. If you set download=true then you will just get the decoded bitstream

$curl "http://localhost:8080/datacapturestore/DefaultCaptureStore/12345?download=true"
HelloWorld

You can also download an ExceptionList if your message has one recorded; set payload=exceptionlist to get the exceptionlist instead of the bitstream. This can also be combined with download=true

$curl "http://localhost:8080/datacapturestore/DefaultCaptureStore/12345?payload=exceptionlist&download=true"

Replaying recorded data

As we already defined a DataDestination called MyDestination above, we have somewhere we can replay to. Again, you can do this via the web browser quite easily, but doing it with curl isn’t hard either.

$curl "http://localhost:8080/datacapturestore/DefaultCaptureStore/12345" -d "replaydestination=MyDestination"

Viewing more about your broker

There are lots of other details about your broker and what is deployed on it available via “http://localhost:8080/admin”. See the REST Api documentation for full details

$curl "http://localhost:7101/admin"

$curl "http://localhost:7101/admin/executiongroup/test/"

$curl "http://localhost:7101/admin/executiongroup/test/properties"


test
ExecutionGroup3adf66ca-3301-0000-0080-814338bb17bb0
ontrue

“http://localhost:7101/admin/executiongroup/test/extendedproperties” will return a very large amount of information.

Several /admin paths also support recurse=true, for example to see everything deployed on your broker use ”http://localhost:8080/admin?recurse=true”

This was only a brief introduction to Record and Replay in WMB 8, and there is much more detail in the normal documentation. Hopefully a future update to WMB will bring more power and functionality to this area, as well as some more database support. Obviously I can’t say what will be added, let alone when, but I hope you get some value out of what is there currently.

edit 16/01/12 – corrections as supplied by Sebastian

Message Broker v8 with MySQL

Anton Piatek — Thu, 15 Dec 2011 09:22:30 +0000

Now that you know that you can use Message Broker v8 on Ubuntu for development, you may be wondering if you can use some of the databases that are more common on Linux, such as MySQL. You can!

IBM WebSphere Message Broker now uses UnixODBC to connect to many databases. Though not supported by IBM, you can actually connect to MySQL from Message Broker using UnixODBC.

If you are on Ubuntu, you can install the MySQL UnixODBC dirver and the MySQL server with apt-get:

apt-get install libmyodbc mysql-server

Your environment variable $ODBCINI should point to your odbc.ini, mine points to /var/mqsi/odbc/odbc.ini and the base file was copied from /opt/IBM/mqsi/8.0.0.0/ODBC/unixodbc/odbc.ini

You need to set a database definition for a mysql database, something like the following (assuming “wmb” is the name of your MySQL database created later)

[MYSQL]
Description     = MYSQL DB
Driver          = MySQL
Database        = wmb
Server          = 127.0.0.1
Port            = 3306

You also need to make sure your $ODBCSYSINI points to the folder (not the file) where your odbcinst.ini file has been copied to. Mine is set to /var/mqsi/odbc and the odbcinst.ini file was copied from /opt/mqsi/8.0.0.0/ODBC/unixodbc/odbcinst.ini, you also need to add a definition to explain to UnixODBC how to connect to a mysql database

The following should be enough to get UnixODBC working with MySQL

#Sample from package libmyodbc
[MySQL]
Description     = MySQL driver
Driver          = /usr/lib/odbc/libmyodbc.so
Setup           = /usr/lib/odbc/libodbcmyS.so
CPTimeout       =
CPReuse         =

Now create a mysql database (mysqladmin create wmb). You may need to GRANT some permissions to your user in MySQL, or add yourself to the mysql group and logout and back in again.

You can now check the database connection using the WMB mqsicvp command, which will also describe which database functions are available:

mqsicvp -n MYSQL -u anton -p password

BIP8270I: Connected to Datasource 'MYSQL' as user 'anton'. The datasource platform is 'MySQL', version '5.1.58-1ubuntu1'.
===========================
databaseProviderVersion      = 5.1.58-1ubuntu1
driverVersion                = 05.01.0006
driverOdbcVersion            = 03.51
driverManagerVersion         = 03.52.0002.0002
driverManagerOdbcVersion     = 03.52
databaseProviderName         = MySQL
datasourceServerName         = 127.0.0.1 via TCP/IP
databaseName                 = wmb
odbcDatasourceName           = MYSQL
driverName                   = libmyodbc5.so
supportsStoredProcedures     = Yes
procedureTerm                = stored procedure
accessibleTables             = No
accessibleProcedures         = No
identifierQuote              = `
specialCharacters            = None
describeParameter            = No
schemaTerm                   =
tableTerm                    = table
sqlSubqueries                = 31
activeEnvironments           = 0
maxDriverConnections         = 0
maxCatalogNameLength         = 192
maxColumnNameLength          = 192
maxSchemaNameLength          = 0
maxStatementLength           = 8192
maxTableNameLength           = 192
supportsDecimalType          = Yes
supportsDateType             = Yes
supportsTimeType             = No
supportsTimeStampType        = No
supportsIntervalType         = No
supportsAbsFunction          = Yes
supportsAcosFunction         = Yes
supportsAsinFunction         = Yes
supportsAtanFunction         = Yes
supportsAtan2Function        = Yes
supportsCeilingFunction      = Yes
supportsCosFunction          = Yes
supportsCotFunction          = Yes
supportsDegreesFunction      = Yes
supportsExpFunction          = Yes
supportsFloorFunction        = Yes
supportsLogFunction          = Yes
supportsLog10Function        = Yes
supportsModFunction          = Yes
supportsPiFunction           = Yes
supportsPowerFunction        = Yes
supportsRadiansFunction      = Yes
supportsRandFunction         = Yes
supportsRoundFunction        = Yes
supportsSignFunction         = Yes
supportsSinFunction          = Yes
supportsSqrtFunction         = Yes
supportsTanFunction          = Yes
supportsTruncateFunction     = Yes
supportsConcatFunction       = Yes
supportsInsertFunction       = Yes
supportsLcaseFunction        = Yes
supportsLeftFunction         = Yes
supportsLengthFunction       = Yes
supportsLTrimFunction        = Yes
supportsPositionFunction     = Yes
supportsRepeatFunction       = Yes
supportsReplaceFunction      = Yes
supportsRightFunction        = Yes
supportsRTrimFunction        = Yes
supportsSpaceFunction        = Yes
supportsSubstringFunction    = Yes
supportsUcaseFunction        = Yes
supportsExtractFunction      = Yes
supportsCaseExpression       = Yes
supportsCastFunction         = Yes
supportsCoalesceFunction     = Yes
supportsNullIfFunction       = Yes
supportsConvertFunction      = No
supportsSumFunction          = Yes
supportsMaxFunction          = Yes
supportsMinFunction          = Yes
supportsCountFunction        = Yes
supportsBetweenPredicate     = Yes
supportsExistsPredicate      = Yes
supportsInPredicate          = Yes
supportsLikePredicate        = Yes
supportsNullPredicate        = Yes
supportsNotNullPredicate     = Yes
supportsLikeEscapeClause     = Yes
supportsClobType             = Yes
supportsBlobType             = Yes
charDatatypeName             = char
varCharDatatypeName          = varchar
longVarCharDatatypeName      = long varchar
clobDatatypeName             = N/A
timeStampDatatypeName        = N/A
binaryDatatypeName           = binary
varBinaryDatatypeName        = varbinary
longVarBinaryDatatypeName    = long varbinary
blobDatatypeName             = N/A
intDatatypeName              = integer
doubleDatatypeName           = double
varCharMaxLength             = 0
longVarCharMaxLength         = 0
clobMaxLength                = 0
varBinaryMaxLength           = 0
longVarBinaryMaxLength       = 0
blobMaxLength                = 0
timeStampMaxLength           = 0
identifierCase               = Mixed
escapeCharacter              = \
longVarCharDatatype          = -1
clobDatatype                 = 0
longVarBinaryDatatype        = -4
blobDatatype                 = 0

BIP8273I: The following datatypes and functions are not natively 
supported by datasource 'MYSQL' using this ODBC driver:
Unsupported datatypes: 'TIME, TIMESTAMP, INTERVAL' Unsupported functions: 'CONVERT'

Examine the specific datatypes and functions not supported
natively by this datasource using this ODBC driver.
When using these datatypes and functions within ESQL, the
associated data processing is done within WebSphere Message
Broker rather than being processed by the database provider.  

Note that "functions" within this message can refer to functions or predicates. 

BIP8071I: Successful command completion.

There – done! Now you can use MySQL with WebSphere Message Broker on Ubuntu. Don’t forget to set the database password with mqsisetdbparms before trying to actually connect from your database nodes.

Also, the name of the database according to WMB is “MYSQL” as that is what I defined in the odbc.ini. The name of the database according to mysql is “wmb”. They don’t need to be different, but it might help having unique names if you are trying to track down a database connection issue.

Of course, the mysql database server does not need to be local to the broker – The mysql odbc driver should allow you connect to any accessible tcpip mysql server.

WebSphere Message Broker on Ubuntu

Anton Piatek — Wed, 14 Dec 2011 09:22:13 +0000

WebSphere Message Broker v8 now supports Ubuntu for development systems (i.e. not production use) - http://www-01.ibm.com/support/docview.wss?uid=swg27023600#Ubuntu

I have been running MQ and Message Broker on Ubuntu and Debian since shortly after I joined IBM in 2005, and it seems there are lots of other people doing this too despite it not being a supported platform before now.

Lots of people have advice on how to install MQ and WMB, and it is worth mentioning them in case you have problems.

http://www.reidmorrison.com/linux/install-websphere-mq-on-ubuntu

http://www.gefira.pl/blog/2010/07/03/websphere-mq-and-ubuntu-howto/

http://blog.herbert.groot.jebbink.nl/2009/06/wmb-61-ubuntu-904.html

The best advice I can give for installing MQ and WMB on Ubuntu is:

change the /bin/sh symlink to point to /bin/bash – MQ Doesn’t like installing with dash as the default shell.

use rpm to install MQ – Alien is a bit of a hack, and does not work well. You will need to use the “–force-debian” flag on rpm to make it install.

One other thing which might help is to run the mqlicense.sh script with the ‘-console’ flag as it may not find your X applications properly.

Some user’s have noticed that chown on Debian and Ubuntu strips the setuid bit from the binaries (Debian and Ubuntu consider leaving setuid set on an executable when you change it’s owner a security flaw, whereas Redhat and SuSE appear not to) so you may need to fix the permissions (best to check the permissions of the same level of MQ from a RHEL or SLES box and set them the same) though I have not seen this with recent versions of MQ.

Message Broker v8 installs quite happily on Ubuntu. The only issues that I know of are that some of the eclipse based gui applications do not draw everything correctly. This is a known eclipse-GTK bug, and is more common on releases after Lucid Lynx (10.04). A workaround is to set the environment variable GDK_NATIVE_WINDOWS=1

Update 05/01/12 – I have just noticed that the script ‘mqsicreateworkpath’ which is used to initialise /var/mqsi correctly still uses ksh. Either install ksh on your system or edit the script to say bash in the first line instead of ksh (it should work fine then)

Frequently Asked Questions about the GNU GPL

Anton Piatek — Thu, 08 Dec 2011 11:18:45 +0000

I have been flicking through the following FAQ on the GNU GPL: Frequently Asked Questions about the GNU GPL – GNU Project – Free Software Foundation (FSF). I am trying to understand the risks that we are told about regularly in IBM of working with/on Open Source code.

I have to say that most of the risks are rather undefined, and no real examples are given. I am having trouble figuring out why IBM is so scared of its employees working with Open Source, and yet at the same time encouraging it (I run Debian on my laptop, and in fact run a Debian server distributing internal tools packaged specially for other Debian users).

The one big concern is of course patents. Personal views on software patents aside, if I were to code a patented idea into some GPL’d code and release it (as an IBM employee) then quite likely that patent will be considered GPL’d and could be used by others – the extent of patents in GPL’d code is untested in court, hence the reason for v3 of the GPL, which will probably explicitly state that any patents are given away under the terms of the GPL if in GPL’d code.

However, even if not modifying the code there is still a huge concern about working with it. Even releasing code that is dynamically linked against a LGPL’d library is incredibly hard to get approved. Why should it be? The FAQ above makes clear that if the library is LGPL’d, then dynamic linking (i.e. using a JAR file in Java) means that the library source must be available and some reverse engineering of the interaction between the two is allowed (in case someone wanted to rewrite the library). Your code does not have to be Open Sourced. Is it simply that shipping the source code is too much effort? Surely if I use a prebuilt JAR, and dump a zip of the corresponding source of a LGPL’d library in the zip I am shipping then that is it. I am complying with the terms of the GPL.

The only thing I can think of is the other side of Patents – somebody else has a patent on something in the GPL code, but the authors of the GPL code were not aware of this. If IBM were to ship GPL’d code that a competitor had a Patent on part of then indeed they would be liable for infringement, not just the author of the code. Is this risk likely? Well, SuSE have just signed a deal with Microsoft for this reason, but RedHat and Ubuntu have declined the deal. Maybe with the GPLv3 this will be less of a problem, as if another company releases GPL code then they will be explicitely releasing any patents on it too.

What if the patent holder is another third party. IBM could still be in trouble if a competitor finds they have a patent on some GPL’d code we are shipping. They won’t bother suing small linux vendors, but IBM has lots of money to claim from, that would be worth going to court for. Could they claim only for IBM and not all the little guys who have already been using it?

Is it too much risk to work on the fact that we could always change the code to avoid the patent at a later date?

Kindle-ing a desire to read

Anton Piatek — Wed, 07 Dec 2011 14:12:56 +0000

This post was originally drafted in May when I actually bought my kindle – I finally got round to actually publishing it…

Earlier this year I finally gave in. I felt a moment of regret about buying into an enormous DRM machine as I clicked purchase on my new Amazon Kindle 3G. I felt a little dirty because of the DRM. Still, I wasn’t buying Sony so I can live with it.

Then theft next day it arrived. So simply packaged, entirely in recycled cardboard, that I forgot about that dirty DRM feeling and tore open the packaging. I pulled out my new kindle and just sat there admiring the instructions showing on the screen. A screen which could show that image for months without charging. It was beautiful.

DRM aside the kindle is an incredible thing.

There are a few hiddenkindle gems including shortcut keys, minesweeper and how to change the screensaver pictures at the following site: From down the hill : Kindle 3 Keyboard Shortcuts et al.

Since then I have been averaging just over a book a month. Some were free, some were £0.71, most are around the £5/£6 mark and one was nearly £10. DRM aside, I have been reading way more than I normally do, as getting to the end of a book is not such an obstacle. The hardest part is choosing what to read next, but while in the middle of a series it is only a 15 second delay to get on to the next book.

I am looking forward to UK lending and eLibraries which support Kindle. Though even without that, I am still reading more than I ever used to and am really loving the fact that I have so much available to read with so little effort. In some ways it is slightly worrying how easy it is to purchase a new book, but at least they are generally cheap.

Upgrading cyanogenmod

Anton Piatek — Mon, 05 Dec 2011 13:48:00 +0000

I previously wrote about rooting a HTC Desire and installing CyanogenMod 7. A month or so ago, a new CyanogenMod rom was released, and I finally found the time to upgrade it.

Of course I first did a full backup, as described in my previous post, using the recover mod.

I first tried updating the rom via RomManager, which is apparently the preferred way to upgrade CyangonMod roms. Updating the Clockwork Recovery Mod went fine, and the app allows very easy downloading of alternate roms, however actually intalling did not go so well. It failed due to some error in the upgrade script. I decided to simply report the error and move on to a manual upgrade.

Manually updating was pretty much the same as installing the roms in my previous post. Using the reovery mod I installed the new Rom from the zip which was downloaded via RomManager, followed by the Google Aps zip. Then I cleared the Dalvik cache, as I wasnt sure if it would interfere (it doesn’t hurt to clear it anyway).

The update was quite smooth, and after a reboot or two nearly everything was working as before, though it took a while for all the app icons to come back on my desktops. As I did not reset any application data I did not have to reinstall any of my apps or restore the data.

The only issue I found is that the calendar would crash upon starting. I installed CatLog, which allows you to view the android system logs, and very quickly found a stack trace from the calendar app saying it could not find a database column.

I guessed that the storage format must have changed, and as it is all synced from my Google calendar anyway, I thought I would just clear the data and let it resync. Under System->Applications I found the app CalendarStorage, and simply deleted all its data. After a moment to resync from the cloud, everything looks fine.

The latest CyanogenMod 7.1 seems to have some fixes I have been waiting for. In particular, I quite liked the dual lockscreen. A shorter timeout for a swype unlock, where the lockscreen has a music widget. The longer timeout is the unlock pattern. This was not possible in the older CyanogenMod 7.0.3 as a bug seemed to only show the pattern unlock screen.

There is quite a list of new fixes, but I don’t think I needed any of them. One new thing I noticed is that holding down the power button gives not only the usual shutdown/reboot options, but also a screenshot option, which is quite useful and removes the need for a separate app to do this.

It feels like a new house!

Anton Piatek — Thu, 03 Nov 2011 09:55:49 +0000

We had some trees removed yesterday, mostly because of their proximity to the house, but they did block a lot of light. Some of the before and after shots really give a striking contrast of how the trees change the property.

We need some time to get used to the new look, but will probably end up planting something, probably some trees (oh, the irony of getting rid of trees to plant some), though certainly nothing like the ones we had. Something smaller and more decorative maybe, if it doesn’t block too much light. Of course keeping it pruned is probably the most important thing to stop them overgrowing like the old ones.

There are also a few shots partway through:

Reduce wordpress comment spam with .htaccess rules

Anton Piatek — Sun, 02 Oct 2011 07:33:36 +0000

I get quite a lot of spam comments on my wordpress blog, and even with things like akismet to try and detect and block spam I still get a lot. Some people have suggested using a mod_rewrite rule in apache to detect very obvious attempts to post unsolicited comments. The basic idea is that if anyone tries to submit a comment and they have not set a referrer in the HTTP headers of your site, then they probably aren’t doing anything near normal browsing. Also, a blank user agent is also only common if you have gone to the effort of not setting one.

Based on the mod_rewrite hack from http://www.wprecipes.com/reduce-spam-on-your-wordpress-blog-by-using-htaccess, I have my own slightly different version


  RewriteEngine On

  RewriteCond %{REQUEST_METHOD} POST
  RewriteCond %{REQUEST_URI} .wp-comments-post\.php*
  RewriteCond %{HTTP_REFERER} !.*YOURSITE\.COM.* [OR]
  RewriteCond %{HTTP_USER_AGENT} ^$
  RewriteRule .* / [R=301,L]

This basically rewrites the url back to / (edit the last line if you want to set it to somewhere else). The rule only works if the request is a POST request and for the wp-comments-post.php file, and either the referrer url was not your site or the user agent is empty. I can’t think of anything that sends blank user agents, so this should be pretty safe. All browsers should also send the referrer url, so again this should be pretty safe.

Don’t forget to change “YOURSITE\.COM” to your domain name

Guest presenting on the Ubuntu UK PodCast

Anton Piatek — Wed, 28 Sep 2011 20:41:00 +0000

I was a guest presenter on this weeks Ubuntu UK PodCast. I talked about Southackton, and IBM’s use of Ubuntu on the desktop.
Listen to the podcast at
http://t.co/vVL5M3YJ

Rooting and upgrading a HTC Desire to android 2.3 (CyanogenMod 7)

Anton Piatek — Thu, 23 Jun 2011 20:20:28 +0000

I started writing this as I was actually doing this to my phone. The problem I found with most existing instructions is they seem to slightly contradict other instructions you find. Many have steps which are not well laid out and often don’t give a reason for doing things which can be a bit annoying. Often the instructions are from ancient Android OS version days, so steps like installing hboot versions, and recovery roms are now all covered with the one unrevoked3 step, but not all instructions are clear that this is the case.

These instructions are heavily based on the instructions from CyanogenMod 7 for HTC Desire: http://wiki.cyanogenmod.com/wiki/HTC_Desire_(GSM):_Full_Update_Guide but also some other sites I found along the way, often for other phone models so only parts apply. I also had some direct feedback from various IRC channels.

The following links were useful to me, so I mention in case you want to read some more:
http://forum.xda-developers.com/forumdisplay.php?f=628
http://forum.xda-developers.com/showthread.php?s=ea2964d1e59f99d394ca758037ee6bc1&t=696189
http://forum.xda-developers.com/showthread.php?t=788044

1. Preparations

The first thing that may be of use, especially if you are checking other instructions, is to grab a few version numbers of the rom levels on your phone:

power off, hold vol down and power – get hboot version page, includes radio version
mine was hboot 0.93.0001
and my radio version is 5.11.05.27
hboot version shouln’t matter for unrevoked3 though unless on very new phone like EVO4
You can also check the radio version from “Settings->About Phone->Software information”.
You should see “Basebane version”, mine is 32.49.00.32U_5.11.05.27
The last part of that, after the _, is the radio rom version

2. Rooting your phone

Go to http://unrevoked.com/recovery/ and download the rooting program.

This uses an exploit in the phone to install a new recovery rom, the “clockwork recovery” rom, which can then be used to do complete phone backups and install new roms.

You will need the “usb debugging” enabled on the phone, and the phone connected only to charge, not as disk access or tethering

It is straight forward on linux (run the downloaded file, and follow the instructions on screen), but I believe on windows there are soe extra drivers to install so check the unrevoked site for more instructions if you are on windows. .

I tried unrevoked3 with only 14mb available, and got “internal error: installing package failed”. Getting the space up to 25mb and it worked fine (I just uninstalled the gmail updates)

3. Backup your current OS (nandroid backup)

Reboot into the hboot screen as in step 1 for version numbers, then use use vol up/down to select “recovery”. The power button acts as the “ok” button here.

This boots the clockwork recovery rom which was installed when you ran unrevoked.

Then use trackball to select backup.

This does a complete backup to SD, also referrred to as a “nandroid backup” which should include the system OS, all installed apps and user data.
You probably need as much space as your phone has on internal memory (512mb for desire).
This will take a little while, so do it on power or a full battery.

The backup should appear in your sdcard in /sdcard/clokworkmod/backups/

Then press back and select the reboot option.

4. Backup apps and app data for migration

Why do you need this if you just did a complete backup? Well, when you install the new OS you will be erasing everything from the internal memory, so this backs up your apps so you can install them again, and also the app-data so you don’t lose it all.

I used the free version of “Titanium Backup” but “Astro File manager” has been suggested too, and it looks like “Gemini App manager” can do this as well.
It is pretty straightforward, but see http://www.incredibleforum.com/forum/htc-incredible-hacks/7893-titanium-backup-how.html if you want specific steps.

5. Backup your whole SD card

Ok, maybe I am paranoid, but at this point I plugged in my phone via USB and zipped up the entire SD card onto my computer. This way I have a backup of my SD card, which includes all the above backups, so if anything goes badly wrong I still have the restore files

I had to disable the usb debug mode to get the disk to mount

6. Update radio rom

Before applying the cyanogenmod, or any other, you may need to update your radio rom version.

According to http://wiki.cyanogenmod.com/wiki/HTC_Desire_(GSM):_Full_Update_Guide the version of mine is good enough for CM7, so I didn’t update it.

7. Now install new rom

Installing the mod is straightfowrard (there are instructions in the first link of this page).

Basically download the mod, and google apps if you want it, and drop the zip files in the root folder of the SD card.
Then use the clockwork recovery mod as we did earlier and select “Wipe data/factory reset” to clear the flash (I have seen reports that this may not be required, but as it is such a large change of version numbers, I decided to do a full wipe), then “Wipe cache partition” to clear that too.

Then simply select “install zip from SD card” and install the new mod, and google apps if you wanted it.

Alternatively, you can use Rom Manager to do it as an app.

After the reboot, use your chosen backup program to then restore all your apps and data, however read the next step first as you may find space to still be an issue.

8. Space savings with CyanogenMod 7

After rebooting into the new OS, I had over 100mb free for base image (out of 148mb capacity reported by OS). 83mb free after installing a similar set of google apps from the market.

After then restoring all my backed up apps and data I found out I only had 25mb free, which is only about 10mb better than on the original android 2.2 from HTC.

You will probably find you have to install an app to force apps to install on SD card by default. Note that CyanogenMod 7 has an option to force all apps there by default, but it will put apps with widgets on there, which breaks the widgets.

There is a way to get even more space for apps with CM7:

All you have to do is add an ext4 partition to your SD card and then install “simple2ext” from the android market.

This is a feature only available on the CyanogenMod 7 roms, and allows you to move all apps, app data, and caches to SD card. There is a performance hit for this, but it seems worth it.

I used a microSD adapter to plug in my SD card into my PC and used “gparted” on Ubuntu (a Live CD will do fine) to shrink the current partition and add an ext4 one
without losing data (but you backed up above in any case, right?)

I created a 1GB partition and after installing simple2ext, I set it to move my apps and dalvik cache to the ext4 partition.

After a reboot I now have loads of space for apps to install to, about 80mb free despite installing another dozen apps without bothering to move them to SD.

To help you decide what to move, the Dalvik cache appears to be a binary cache for the Android implementation of Java. There is a performance hit to moving anything to SD unless you have quite a fast card (http://forum.xda-developers.com/showthread.php?t=1175707 recommends minimum class 4, better yet a class 6). Mine is class 2, and I found that moving the application data seems to
cause the phone to be very noticably slower, so it is probably best avoided unless you have a fast SD card.

Moving the Davlik cache did slow things down a little, but mostly only the boot time and first launch of an app. I decided that for an extra 80mb of space that I could live with that (I have also just ordered a new class 6 SD card, so that should help).

Conclusions

Rooting, and modding was actually incredibly simple. It sounds so scary, but I think it is actually very, very hard to mess it up to a point where you cannot just overwrite with a new mod (or your backup) and start again. Once you get to the nandoid backup point you have a very good recovery strategy!

I have loads more space, which is good, because counting the bytes is annoying. I now don’t have to worry about if I have space to install something.

You do lose the HTC apps and widgets, but I found I could live without them fairly easily (I haven’t used peep since I discovered TweetDeck, and that was my most used HTC app). If you miss the clock widget, search for “sense analog glass clock” on the android market, that is a nice replacement for me. “glass widgets” is a replacement calendar widget, which I now actually prefer.

The battery manager in the new rom is awesome compared to the old 2.2 one. It really shows where the battery is going. I think I possibly get longer battery life with this rom too.

I much prefer the android 2.3 native keyboard to the HTC sense one.

There are some niggles – I have a few gui glitches which cause me to reboot the phone every few weeks, but I am hoping they will be fixed in the next stable build (I have not been brave enough to try the nightly builds :-s )

Overall, I quite like the new UI though it did take some getting used to. The fact I dont have to worry about space is the biggest reason I will keep this rom, though it does sound like starburst rom which is much closer to vanilla HTC sense branded android also allows the “data2sd” hack to get more space

Time is short. (Or why piracy is not the problem)

Anton Piatek — Tue, 10 May 2011 19:46:41 +0000

We are constantly being bombarded by claims that the music and film industries are being destroyed by rampant piracy eating away at their profits. This whole argument really annoys me, and partly because of the following really well illustrated point.

Click to load

I pay for a DVD and then waste loads of time being told not to pirate DVDs. If I had pirated the DVD, I wouldn’t have had to put up with all that.

So how does that fit with a title of “Time is short”? Well… You may have noticed that I did not write a single blog post last month – I haven’t had the time. I also haven’t had the time to go to the cinema, the last film I saw was the penultimate Harry Potter and it not only cost an arm and a leg, but a drink and some nibbles nearly cost me the other arm.

I also have not bought much music in a while – I haven’t heard much I particularly like on the radio, and haven’t really had time to go out looking for anything new, so just stuck with my iPod. If I did buy music, it would probably be a digital download for only a few pounds per album.

You might have guessed that I haven’t bought a DVD in a while – My lovefilm subscription gives me two DVDs a month to watch, and that keeps me busy enough with a little trashy tv.

I have bought a few X-Box games, and am likely to buy some more – Quite shocking really, as they are about £40 or £45 each. So that is eating up a lot of my time as I work my way through Crysis 2 (which is a truly awesome game) and try to resist handing over my wallet Portal 2.

I am also hacking away at Arduinos, rebuiling my media PC and hanging out on twitter and facebook. I wish I could say I even read some of my friends blog’s, but sadly the only items in my reader feed being read are lolcats, failblog, dilbert and xkcd.

As I have already said my blog barely gets updated as it is.

I bought a kindle a few months ago, and so have been reading a lot of books, which really is fantastic and of course consumes huge amounts of my time.

And next month (or possibly sooner) I will be a father, so that will take away all the free time I had for all the above activities!

So where am I supposed to find the time to watch all these films and DVDs that the industry says are being pirated? Piracy is not the problem – Crap films, high prices and, yes, time. They are what is causing me to spend less money on films and music. Instead I am spending more on games, books, gadgets, toys, and soon children.

Cactus flowers

Anton Piatek — Sat, 26 Mar 2011 16:22:34 +0000

I finally got round to posting some photos I took last year of the cactus I have flowering. It has only done it twice in the ~6 years I have had it, and this time it had quite a few flowers. The macro photos of it came out the best, helped by sheets of A4 paper as light reflectors (and in some photos backgrounds). I cannot remember if I used a wireless flash or not, but I suspect I probably did.

Blue Fusion 2011

Anton Piatek — Tue, 15 Mar 2011 08:24:26 +0000

As it is National Science and Engineering week, IBM Hursley are running Blue Fusion 2011 - The official press release summed it up quite well, and also contains a list of this year’s activities:

HURSLEY, U.K. - 11 Mar 2011: More than 450 students from 80 schools across the South of England will be celebrating National Science and Engineering Week at IBM’s 16th annual Blue Fusion event. The students will need to use quick thinking, scientific knowledge and teamwork in a number of fun activities, designed to inspire and encourage young people to develop their interest in Science, Technology and Engineering.

“Communication” is the theme for National Science and Engineering Week 2011 and volunteers from IBM Hursley, Europe’s largest software development lab, have put together a wealth of fun, inspiring and engaging activities.

Each day, teams from 16 schools will participate in a variety of specially-designed activities promoting innovation, teamwork, problem solving and communication skills. Students solve puzzles and technological challenges that explore basic concepts in some of the most interesting areas of modern technology. The activities will include building network communications to cracking codes to designing an ecologically friendly house. The students will experience first hand how science benefits our everyday lives and how instrumented, interconnected and intelligent technology can help build a smarter planet.

Approximately 200 IBM employees volunteer their time and experience to Blue Fusion, many of whom are recent graduates who can share their passion and experience of working in the industry with the next generation. Local dignitaries will be attending the event throughout the week to see the impact and success of Blue Fusion in inspiring young people into Science, Technology and Engineering.

I look forward to hosting schools on Thursday – it has been fun being involved for the last five years, so should be great this year too! I wrote briefly about Blue Fusion and Bright Sparks in 2007 (I can’t remember if that was the year I organised and ran Blue Fusion with Dan)

There are one or two videos about Blue Fusion which really show what the event is about:

Infra-red photowalk

Anton Piatek — Tue, 01 Feb 2011 18:46:04 +0000

It seems I never put the photos from the Southampton Photo Walk 2010 on my blog… Oh well, better late than never I suppose.

The blog page that talked about last year’s walk seems to have disappeared, but there is still a flickr group

Here’s what I took (in with my infra-red camera)

BT Infinity – Initial Speed

Anton Piatek — Tue, 01 Feb 2011 12:37:30 +0000

The engineer has just left and I now have BT Infinity broadband.

The speed below will probably fluctuate (a lot) over the next few days, but 36Mbit/s is pretty impressive:

Certainly an improvement from my previous Virgin broadband

Arduino talking to an I2C LCD display

Anton Piatek — Mon, 17 Jan 2011 19:33:05 +0000

I had a spare Matrix Orbital 20×4 character LCD floating about so I decided to try and connect it up to my Arduino. The display is a LK204-25, which is a brilliant little serial display, with support for a 25 button keypad and six outputs that can be used to control power to something else.

Being a serial display, I could just write to it from my Arduino using the serial interface, but that would mean that I lose the ability to communicate with my Arduino from my computer over serial. Fortunately the display also supports I2C as an interface (some similar, but newer, models also contain a 1-wire interface and a temperature sensor built in).

It turns out that getting the display to work with my Aruino Uni, using I2C is rather trivial. The display needs power, that is easy enough, and then there are just two wires to connect from the Arduino – Analog pin 4 for is SDA, analog pin 5 is SCL. Connect those two up to the display (the display docs show which pins SDA and SCL are) and you are ready to write the code to control it.

It turns out that I2C has fantastic support in Arduino, Wire.h contains all you need. The code below will read a character from the serial connection from your computer and write it to the display. Easy, eh?

// Note the space after the . in the imports below - This blog seems to object to not having it there
#include < wire.h>
// default MatrixOrbital lcd address (0x5C) converted from 8bit to 7bit
#define LCD (0x2E) 
 
void setup() {
  Serial.begin(9600);
  Wire.begin();
 
  // Initialise display with clear command
  Wire.beginTransmission(LCD);
  Wire.send(254);
  Wire.send(88);
  Wire.endTransmission();
}
 
void loop() {
  if (Serial.available() > 0) {
    char c = Serial.read();
    Serial.write(c);
    Wire.beginTransmission(LCD);
    Wire.send(c);
    Wire.endTransmission();
  }
}

process forking in perl

Anton Piatek — Thu, 06 Jan 2011 10:14:19 +0000

This is a very geeky topic, but I was having a conversation at work today with someone who was trying to do several really simple things in parallel in perl because they didn’t want to wait several minutes for each one to finish. They had knocked up a change to the existing perl, but had tried to do it using perl threads. Normally, that would be fine for most boxes, but when you are running rather old perl on various unixes, including under USS on the IBM z/OS Mainframe, you tend to find out that perl was not compiled with thread support (and recompiling perl for the mainframe is not the easiest thing in the world).

I suggested that they simply use fork() to do the same thing with multiple perl processes, and having not done this in quite a while I tried to search for a good example but couldn’t find anything that looked both complete and simple enough to explain quickly – Hence this post…

#!/usr/bin/perl 
use strict;
use warnings;
 
sub main
{
  my @children;
 
  foreach my $i ( reverse(1..3) )
  {
    my $pid = fork();
 
    if( $pid )
    {
      #If $pid is non zero, then the parent is running
      print "PID $pid forked ($i)\n";
      push(@children, $pid);
    }
    else
    {
      # Else we are a child process ($pid == 0)
      my $rc = child($i);
      exit($rc);
    }
  }
 
  foreach my $n (@children)
  {
    my $pid = waitpid($n,0); # waitpid returns the pid that finished, see perldoc -f waitpid
    my $rc = $? >> 8; # remove signal / dump bits from rc
    print "PID $pid finished with rc $rc\n";
  }
}
 
sub child
{
  my ($arg) = @_;
  print "$arg: start\n";
  sleep $arg;
  print "$arg: end\n";
  return $arg*2;
}
 
main();
exit;

And the output of running the above code is as follows:

PID 16767 forked (3)
PID 16768 forked (2)
PID 16769 forked (1)
3: start
2: start
1: start
1: end
2: end
3: end
PID 16767 finished with rc 6
PID 16768 finished with rc 4
PID 16769 finished with rc 2

It is worth remembering that because each process gets an entire copy of the memory, you have to do more clever things to actually get the processes to talk to each other – but if all you want to do is run something and get a return code back, the above example should get you going pretty quickly.

For what it is worth, the example at https://wiki.bc.net/atl-conf/pages/viewpage.action?pageId=20548191 isn’t too bad actually, and compares perl threads with process forking, and a php example.

Fix your Arduino Uno for Linux

Anton Piatek — Mon, 13 Dec 2010 17:25:46 +0000

Are you using an Arduino Uno on Linux? If so, you may have noticed that writing to the serial port in a loop can cause the Arduino Editor/Programmer software to appear to lock up, or even Linux having trouble using the serial port for your Uno.

It turns out there was a bug in the USB->Serial firmware on the chip which caused this [1], and it is quite easy to update your arduino to fix it. According to the author of the USB->Serial firmware, it is not possible to break your Arduino permenantly doing this [1], but I can’t promise anything (though it worked fine for me and others)

From https://github.com/arduino/Arduino/tree/master/hardware/arduino/firmwares/arduino-usbserial/ download your hex file for your board, i.e. either Arduino-usbserial-mega.hex or Arduino-usbserial-uno.hex. As it is github, click on the file in the list then right click on “raw” and click “Save as“.

The md5sum of the Arduino Uno file from the 4th December is 8e01ee236e70bbea43f7eb4e11c9688a

You will need the dfu programmer tools, which are conveniently in Lucid/Universe, so just run

sudo aptitude install dfu-programmer

You need to do a hardware-reset to put the chip into the correct DFU programming mode, which is simply best explained with the picture at http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1285962838/10#10 (you will need to register on the arduino site to login, but a picture speaks a thousand words)

The Uno’s use a AT90USB82 serial-usb chip, so we then reflash the firmware with the following commands

sudo dfu-programmer at90usb82 erase
sudo dfu-programmer at90usb82 flash --debug 1 Arduino-usbserial-uno.hex
sudo dfu-programmer at90usb82 reset

Finally, just disconnect the USB lead and reconnect to reset the device back to normal mode.

Done – You no longer have to worry about writing to your serial port on your Arduino.

References:
[1] http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1286088093/38#38 – Author of Arduino USB stack talks about
[2] http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1286088093/34#34 – First person to find fix and author of instructions that these are heavily based on

Arduino 1-wire temperature sensors

Anton Piatek — Mon, 13 Dec 2010 08:42:01 +0000

So you have played a bit with your Arduino, and have heard about other people using it to monitor the temperature, but weren’t sure how they did it – Here is how I did.

Maxim make a very simple 1-wire temperature sensor – the DS18B20 (the replacement to the DS1820). This 3 pin sensor looks just like a transistor, and can work with only two wires, combined data+power and ground. You can put several on the same wire and address them all separately.

The DS18B20 is less that £4 from RS, so not exactly expensive.

Wiring up the 1-wire sensors is fairly simple:

You can power them with either dedicated power wires (exercise for the reader), or use their “parasitic mode” where they use power from the same wire as data. To wire them this way you need to wire pins 1 and 3 (ground and Vcc) from the DS18B20 both to ground. Pin 2 goes to your Arduino (any digital I/O pin), and also 5v DC through a 4.7kΩ resistor (if you have a lot of 1-wire devices, there are reports that you may need to reduce this down to 2kΩ).
If you have long wires you may need to start looking at dedicated power, but certainly for testing parasitic power will be fine.

Stringing multiple 1-wire sensors together is as simple as connecting all the pins up in parallel.

To actually query the devices on your Arduino, you will need the 1-wire and Dallas temperature sensor libraries (unzip them to your Arduino editors’ libraries/ folder).

Examples of using the sensors are all over the place (also examples of querying the ID’s of the devices), or you can try mine below which prints to the serial connection the hex ID’s for each 1-wire device found, and the temperature of each every 10 sec.

// OneWire and DallasTemperature libraries from
//   http://milesburton.com/index.php?title=Dallas_Temperature_Control_Library
// Code based on examples from above and at
//   http://www.hacktronics.com/Tutorials/arduino-1-wire-address-finder.html
// See also http://www.arduino.cc/playground/Learning/OneWire
 
#include 
#include 
 
// Data wire is plugged into pin 2 on the Arduino (can be any digital I/O pin)
#define ONE_WIRE_BUS 2
 
// Setup a oneWire instance to communicate with any OneWire devices
// (not just Maxim/Dallas temperature ICs)
OneWire oneWire(ONE_WIRE_BUS);
 
// Pass our oneWire reference to Dallas Temperature.
DallasTemperature sensors(&amp;oneWire);
 
int numberOfSensors;
 
void setup(void)
{
  // start serial port
  Serial.begin(9600);
  Serial.println("1-Wire DS18B20 example code");
 
  // Start up the library
  sensors.begin();
 
  delay(5000);  //important on linux as serial port can lock up otherwise
  numberOfSensors = discoverOneWireDevices();
  Serial.println();
}
 
void loop(void)
{
  printTemperaturesToSerial();
  delay(10000); //wait 10 sec
}
 
void printTemperaturesToSerial(void) {
  // call sensors.requestTemperatures() to issue a global temperature
  // request to all devices on the bus
  Serial.print("Requesting temperatures...");
  sensors.requestTemperatures(); // Send the command to get temperatures
  Serial.println("DONE");
 
  // Read each of our sensors and print the value
  for(int i=0; i < numberOfSensors; i++) {
   Serial.print("Temperature for Device ");
   Serial.print( i );
   Serial.print(" is: ");
   // Why "byIndex"? You can have more than one IC on the same bus. 
   // 0 refers to the first IC on the wire
   Serial.println( sensors.getTempCByIndex(i) );
  }
 
  Serial.println();
}
 
// Based on http://www.hacktronics.com/Tutorials/arduino-1-wire-address-finder.html
int discoverOneWireDevices(void) {
  byte i;
  byte present = 0;
  byte data[12];
  byte addr[8];
  int count = 0;
 
  Serial.println("Looking for 1-Wire devices...");
  while(oneWire.search(addr)) {
    Serial.print("Found \'1-Wire\' device with address: ");
    for( i = 0; i < 8; i++) {
      Serial.print("0x");
      if (addr[i] < 16) {
        Serial.print('0');
      }
      Serial.print(addr[i], HEX);
      if (i < 7) {
        Serial.print(", ");
      }
    }
    if ( OneWire::crc8( addr, 7) != addr[7]) {
        Serial.println("CRC is not valid!");
        return 0;
    }
    Serial.println();
    count++;
  }
  Serial.println("That's it.");
  oneWire.reset_search();
  return count;
}

I get the following output on the serial connection

1-Wire DS18B20 example code
Looking for 1-Wire devices...
Found '1-Wire' device with address: 0x28, 0xCE, 0x85, 0xBB, 0x02, 0x00, 0x00, 0xC1
Found '1-Wire' device with address: 0x28, 0xEF, 0x7F, 0xBB, 0x02, 0x00, 0x00, 0x5B
That's it.

Requesting temperatures...DONE
Temperature for Device 0 is: 21.31
Temperature for Device 1 is: 21.37

Requesting temperatures...DONE
Temperature for Device 0 is: 21.37
Temperature for Device 1 is: 21.37
...

There you have it – A very quick Arduino temperature sensor, using pretty cheap 1-wire devices. The only problem you have now is how to get the wires everywhere you might want to read the temperature of.

Who came up with eBook pricing

Anton Piatek — Sat, 07 Aug 2010 10:34:09 +0000

Seriously… Who decided how to price eBooks?

A friend at work recently recommended I read Richard Morgan’s Altered Carbon. The Kindle app on android allowed me to find it quickly and download the first 3 chapters free to read – I loved it and decided I wanted to read the rest of it… However when checking the prices I was left the feeling that eBooks are just too expensive for what they are

Play.com £5.99
Amazon UK £5.19
Kindle (Amazon UK) £4.88

I went with Play as their free delivery normally takes about 2 days, so that was great. But seriously, nearly the same price for Kindle? For a book I cannot resell, lend or give away?

Digital books should be much, much cheaper – There is no printing cost involved, so the book should be significantly cheaper, but this is not what I am seeing.

If I choose to keep the book and reread it again in a few years, the price may be fine. But I am more likely to read it once, and then give it to a friend to read. Why should i pay nearly the same price for something I cannot lend, sell, or give to a charity shop?

Also, how on earth does a paper book qualify for no VAT, yet as soon as that book has no paper involved I have to pay VAT?!?

There are lots of Kindle books for £3 or less, and for these I will probably just buy it digitally as its incredibly cheap. But if I wanted to buy a recent release, then I really am paying a lot more for a digital book just to have it early? Why?
I can understand pricing getting lower for older books as they have to compete with people lending books, libraries and second-hand sales, however none of that is possible for DRM protected eBooks, so they should all be priced the same as older releases