Time Tracker application

I have written a little app to keep track what I’m doing all day. It’ll tell you how long you spend in each application – Outlook, IE/Chrome/Firefox, Word… everything. Quite handy really and the results can be quite surprising! Here’s how it looks:

Time Tracking Application

Want to try it out? It’s available here: TimeTracker.exe.

Posted in technical | Leave a comment

Fixing the Remote validation attribute

Using the [Remote] validation attribute in MVC does something cool – it will generate some AJAX for you to call your server-side code for validation. This is great, but what it doesn’t do is call this code on the server side upon postback – unlike the other attributes do. This sucks!

In other words, tagging a property with [StringLength(3)] means that the javascript validation will stop the user from entering a string longer than 3 characters. If a sneaky user came along and turned off javascript, when they submit the form the MVC framework will still validate the string and ensure it’s not any longer than 3 characters. This is cool.

However, if you are using the [Remote] attribute to do some validation that is slightly more clever, like validating that some data entered isn’t already in the database, if the user turns off javascript, the validation will not run automatically on the server. This is deceptively dangerous, because you really expect it to work like the other ones do. You want it to work like this:

better mvc3 remote validation

A simple fix would be “remember to call the same validate logic in your controller too” – but as we all know, developers forget things. Talk about a sucky security hole.

So instead, I’ve written some code that calls the server side remote validation for you within the Model Binding. This means it’s all done seamlessly, all you need to do is check ModelState.IsValid like you normally would. If you screw up your parameters, it’ll fail silently, something that I’m not crazy about, but hey at least it’s consistent with how the javascript validation works (it fails silently too).

The first step is to subclass the [RemoteAttribute] class:

public class CustomRemoteAttribute: RemoteAttribute
{
  public CustomRemoteAttribute(string action, string controller)
    : base(action, controller)
  {
    Action = action;
    Controller = controller;
  }
  public string Action { get; set; }
  public string Controller { get; set; }
}

Strictly speaking this shouldn’t be neccesary. According to this, the RouteData class should be public. But on my machine (MVC3 I think) it’s private. So this is my workaround.

The next step is to use this attribute within my ViewModel:

public class TestPage1ViewModel
{
  [Required(ErrorMessage="Please enter this")]
  [StringLength(3)]
  public string FirstName { get; set; }

  [CustomRemoteAttribute( "ValidateLastName", "Test")]
  public string LastName { get; set; }

  public string Names { get; set; }
}

and then use this viewmodel within my page:

@model MvcApplication3.Models.TestPage1ViewModel

@using (@Html.BeginForm())
{
  @Html.ValidationSummary()

  <fieldset>
    First Name:<br />
    @Html.TextBoxFor(x => x.FirstName)
    @Html.ValidationMessageFor(x => x.FirstName)
    <br /><br />

    Last Name:<br />
    @Html.TextBoxFor(x => x.LastName)
    @Html.ValidationMessageFor(x => x.LastName)
    <br /><br />

    <input type="submit" />
  </fieldset>

  <p>
    Calculated names:
    @Model.Names
  </p>
}

now I need to add the validator code to my controller:

public class TestController : Controller
{
  public ActionResult ViewPage1()
  {
    TestPage1ViewModel vm = new TestPage1ViewModel();
    return View(vm);
  }

  [HttpPost]
  public ActionResult ViewPage1(TestPage1ViewModel vm)
  {
    if (ModelState.IsValid)
    {
      vm.Names = vm.FirstName + " " + vm.LastName;
    }
    return View(vm);
  }

  public JsonResult ValidateLastName(string LastName)
  {
    if (LastName != null)
    {
      if (LastName.ToLower().Trim() == "smith")
      {
        return Json("sorry, I don't like the surname smith",
          JsonRequestBehavior.AllowGet);
      }
    }
    return Json(true, JsonRequestBehavior.AllowGet);
  }
}

And now for the magic! We’re going to define a new custom Model binder that looks for the CustomRemoteAttribute and tries to call the validator using reflection:

public class CustomModelBinder : DefaultModelBinder
{

  /// <summary>
  /// This custom validator checks to see if the property has a
  /// CustomRemoteAttribute on it. If so, it runs the remote
  /// validator function (from here, ie the server side using
  /// reflection) and if it fails it sets the model state error
  /// on the property.
  /// </summary>
  /// <param name="controllerContext"></param>
  /// <param name="bindingContext"></param>
  /// <param name="propertyDescriptor"></param>
  protected override void BindProperty(ControllerContext controllerContext,
    ModelBindingContext bindingContext,
    PropertyDescriptor propertyDescriptor)
  {
    if (propertyDescriptor.PropertyType == typeof(string))
    {
      CustomRemoteAttribute remoteAttribute =
        propertyDescriptor.Attributes.OfType()
          .FirstOrDefault();

      if (remoteAttribute != null)
      {
        List allControllers = GetControllerNames();

        Type controllerType = allControllers.Where(x => x.Name ==
            remoteAttribute.Controller + "Controller").FirstOrDefault();

        if (controllerType != null)
        {
          MethodInfo methodInfo = controllerType.GetMethod(remoteAttribute.Action);

          if (methodInfo != null)
          {
            string validationResponse = callRemoteValidationFunction(
              controllerContext,
              bindingContext,
              propertyDescriptor,
              controllerType,
              methodInfo,
              remoteAttribute.AdditionalFields);

            if (validationResponse != null)
            {
              bindingContext.ModelState.AddModelError(propertyDescriptor.Name,
                validationResponse);
            }
          }
        }
      }
    }

    base.BindProperty(controllerContext, bindingContext, propertyDescriptor);
  }

  /// This function calls the indicated method on a new instance of the supplied
  /// controller type and return the error string. (NULL if not)
  private string callRemoteValidationFunction(
    ControllerContext controllerContext,
    ModelBindingContext bindingContext,
    PropertyDescriptor propertyDescriptor,
    Type ControllerType,
    MethodInfo methodInfo,
    string AdditionalFields)
  {

    string propertyValue = controllerContext.RequestContext.HttpContext.Request.Form[
        bindingContext.ModelName + propertyDescriptor.Name];

    Controller controller = (Controller)Activator.CreateInstance(ControllerType);
    object result = null;
    ParameterInfo[] parameters = methodInfo.GetParameters();
    if (parameters.Length == 0)
    {
      result = methodInfo.Invoke(controller, null);
    }
    else
    {
      List parametersArray = new List();

      parametersArray.Add(propertyValue);

      if (parameters.Length == 1)
      {
        result = methodInfo.Invoke(controller, parametersArray.ToArray());
      }
      else
      {
        if (!string.IsNullOrEmpty(AdditionalFields))
        {
          foreach (var additionalFieldName in AdditionalFields.Split(','))
          {
            string additionalFieldValue =
                controllerContext.RequestContext.HttpContext.Request.Form[
                  bindingContext.ModelName + additionalFieldName];
            parametersArray.Add(additionalFieldValue);
          }

          if (parametersArray.Count == parameters.Length)
          {
            result = methodInfo.Invoke(controller, parametersArray.ToArray());
          }
        }
      }
    }

    if (result != null)
    {
      return (((JsonResult)result).Data as string);
    }
    return null;
  }

  /// Returns a list of all Controller types
  private List<Type> GetControllerNames()
  {
    List<Type> controllerNames = new List<Type>();
    GetSubClasses<Controller>().ForEach(type => controllerNames.Add(type));
    return controllerNames;
  }

  private List<Type> GetSubClasses<T>()
  {
    return Assembly.GetCallingAssembly().GetTypes().Where(
      type => type.IsSubclassOf(typeof(T))).ToList();
  }

}

One last step! Wire up the CustomBinder!

protected void Application_Start()
{
  ...

  ModelBinders.Binders.DefaultBinder = new CustomModelBinder();
}

That should do it! You can now disable javascript, enter “smith” as a surname, and BAM, validation!

You can also use the “AdditionalFields” attribute:

  [CustomRemoteAttribute("ValidateAllNames", "Test",
    AdditionalFields="FirstName,LastName")]
  public string MiddleName { get; set; }

but make sure that you name your fields properly, otherwise it’ll fail SILENTLY !!

Posted in technical | Leave a comment

I am loving these Bundling and Minification libraries for JS and CSS files. Absolute magic. To give you a bit of an idea, today I cleaned up a medium size web application project and went from 27 http requests for every page down to 8! Magic! Decrease load on server, faster response for the user, more fun for everyone :)

Posted in technical | Leave a comment

What programming concepts do I need to master?

The folks at programmers.blogoverflow.com were nice enough to publish an article that I wrote titled “What programming concepts I should master to have a deep understanding of my craft?“. That was nice of them!

Posted in technical | Leave a comment

MVC4, Bundling, IHttpHandler for Html5 Offline Storage

The last couple of months I’ve been working on a new web app. It’s all very cutting edge, using MVC4, HTML5, JQuery, CSS3 – the works! More info to come later once I’ve got it all running.

Anyway I’ve been messing with the Bundling stuff that’s available as part of System.Web.Optimization – and I’m loving it! I can group together all of my CSS and Javascript files into 2 separate minified files. The content expiry is set automatically and a new URL is generated whenever you change one of the files. Now that’s COOL. Web performance is now the way it should be! (Hooray, only took us about 15 years)

I’ve also been looking at a lot of HTML5 features, which are all looking very cool. Microsoft have really let the ball down here and STILL don’t support any of the important ones. Yeah I know they will support them with IE10… but honestly guys… how about backporting them to IE9 or IE8, which is what the majority of people have? No that’d be too handy.

The latest feature that I’ve found great is Offline Web Applications – which means you can pull out your network cable but still access your website! Now that’s cool.

Now come the joys of combining Bundling with an Offline WebApp!

There’s a great article here on how to do it… but of course it doesn’t work. Well, it did work when you use the Visual Studio Development Server – but honestly that’s for chumps, developing against a different web server than your production one is just a dumb idea.

So where I’ve been getting stuck is just implementing my own generic HTTP Handler with MVC4. All I was getting is dumb 404 messages. So after 2 hours I finally figured out that instead of doing the following changes to my web.config:

<system.web>
    <httpHandlers>
        <add verb="GET" path="*/cache.manifest"
            type="namespace.CacheManifestHandler" />
    </httpHandlers>
</system.web>

what I actually needed to do was:

  <system.webServer>
  <handlers>
    ...
    <add name="Test" verb="GET" path="cache.manifest"
            type="namespace.CacheManifestHandler" />
  </handlers>

and VOILA! It works!

Posted in technical | Leave a comment

Mars Curiosity Rover Software and Hardware

The Mars Curiosity Rover has recently landed on Mars! Here’s some info on the hardware and software:

Hardware:

• A BAE RAD 750 single-board computer with a 750Mhz Power PC CPU;

• Two gigabytes of flash memory;

• 256 megabytes of DRAM;

• 256 kilobytes of electrically erasable programmable read-only memory.

• 5 megabits per second from earth until it puts up the main antennae, then it’ll get to 40 megabits.

• If I’ve done my maths right that’s 5 meg a second, not bad for a connection to Mars!

• 14 minute latency. That’s 28 minutes round trip!!  Bit slower than your usual 200ms to google!

• Modern CPU’s will not work properly because of radiation! At the processor is smaller they are more suspectable to protons causing upsets (errors)!

• So they have multiple CPU’s all running the same code on them.

Operating system is named “VxWorks” :

- Has no memory paging to disk, so that the application is completely deterministic.

- So your app code and data must fit inside the RAM on the machine

- Has no processes, but it has tasks, which are similar to threads, but switching between threads has very little overhead compared to linux/windows

- If you give a task priority #1, then all other tasks will be completely on hold

Software:

- Written in C (assembly would be much more error prone)

- 3.5 million lines of code of which 1 million lines hand-coded and 2.5 million generated

- every function call is wrapped around a “reliability library” – if it needs to be super reliable then it’s tested multiple times, if it needs to be fast but not super reliable (eg image
capturing) then there’s a trade off.

Other Cool Stuff
The OS is about to be upgraded, and it’s a big risk!! The current version of the OS has all the code for the entry and landing, and now it needs all the code for roving and research (and instruments etc):

http://www.theregister.co.uk/2012/08/07/curiosity_software_upgrade/

That is one massively awesome project.

Posted in technical | Leave a comment

MVC3, Jquery, Lambda expressions and Entity framework

Lately I have really been getting stuck into MVC3. Initially I wasn’t convinced. The big reason for me wanting to avoid MVC3 was that Webforms provided the perfect solution for having to maintain state across postbacks, something that drove me crazy in the old days. Back then I was doing websites in Perl and JSP’s and it was a nightmare. But with MVC3 you lose all of the lovely viewstate stuff (btw, don’t let anyone tell you that viewstate sucks – if they say that they’re doing it wrong). But now I’m starting to learn jQuery, and things are looking up. I still hate javascript, it still feels like playing in a sandpit with a bunch of 2 year olds, but it can be pretty powerful.

I would say though that the benefits of MVC3 can’t really be attained without a bit of JQuery. I remember 10 years ago trying to write client side validation on the web, and boy it was painful. JQuery and the validation library (that actually works) both make all of that pretty good these days.

I’ve also been using the entity framework, and I’m starting to like it. Yes it gives you less control, yes you can create horrible SQL, but it can save you a lot of time. You can always default back to SQL for bigger queries, but for the simple CRUD I’m starting to love the entity framework. Combining it with AutoMapper (automapper.codeplex.com) and now you’re getting somewhere.

Today I wanted to come across a better solution for converting a List of objects from TypeA to TypeB. Previously I was doing something like:

List things = GetFromDatabaseOrSomething();

List otherthings = new List();

if (otherthings != null)
{
	foreach (ClassA thing in things)
	{
		ClassB other = new ClassB();
		other.propertyA = thing.PropertyZ;
		other.propertyB = thing.PropertyY;
		otherthings.Add(other);
	}
}

That’s pretty long winded. But here’s a nicerer way:


List otherthings =
      (things == null) ? null : things.ConvertAll(x => new ClassB()
            { propertyA = x.PropertyZ, propertyB = x.PropertyY });

That’s more like it! Sure it’s a bit more obscure, but really, it’s pretty simple.

So I’m almost up to date with all of the latest microsoft.net frameworks. What are the main ones that I’ve missed?

Posted in technical | Leave a comment

.NET Entity framework

I have been investigating the entity framework for the past couple of weeks. Yesterday
I asked this question on stack overflow:

entity-framework-is-generating-sql-that-returns-the-entire-table

Now that I’ve got that sorted, I reckon it’s starting to look pretty good.

Let’s say I’ve got a ProjectManager.cs class, and it has the following:

public static ProjectEntity ProjectGet(int ProjectID, string ExecutedBy)
{
ProjectEntity Project = ProjectDAO.Get(ProjectID, ExecutedBy, null);
if (Project != null)
{
Project.Emails = EmailDAO.GetForProjectID(ProjectID, ExecutedBy);
Project.Documents = FileStoreDAO.GetForProjectID(ProjectID, null, ExecutedBy);
}
return Project;
}

I can change it to return exactly the same data, except instead of using the DAO and associated stored procs I can use the entity framework:


public static ProjectEntity ProjectGet(int ProjectID, string ExecutedBy)
{
ProjectEntity project = null;

using (databasecontext context = new databasecontext())
{
Project proj = context.Projects.Where(p => p.ProjectID == ProjectID).FirstOrDefault();

if (proj != null)
{
List projectEmails =
(from pe in context.ProjectEmails
join e in context.Emails on pe.EmailID equals e.EmailID
where pe.ProjectID == proj.ProjectID
select e).ToList();

List emailattachments =
(from pe in context.ProjectEmails
join e in context.Emails on pe.EmailID equals e.EmailID
join ea in context.EmailAttachments on e.EmailID equals ea.EmailID
join f in context.FileStores on ea.FileStoreID equals f.FileStoreID
where pe.ProjectID == proj.ProjectID
select f).ToList();

// Convert all of the entity framework's classes to our normal Entity classes:
project = Utility.ConvertToEntity(proj);
project.Emails = Utility.ConvertToEntityList(projectEmails);
project.Documents = Utility.ConvertToEntityList(emailattachments);
}
}
return project;
}

This is pretty cool. Pros:

* Don’t need to write a stored proc to retrieve data,
* Don’t have to manage the proc in SVN
* Don’t have to worry about the DAO setting the parameters properly for the stored proc –
eg when doing an insert the parameters and types need to be specified properly
* Don’t need a DAO class or function to retrieve data
* Query language is strongly typed. If you modify a table etc it’ll be CAUGHT AT COMPILE TIME.
* Query is very straightforward and easy to read
* Don’t need to keep stored proc up to date or synchronised across databases
* For complicated queries stored procs can easily be used, the entity framework just call it

Cons:
* In theory, DB performance might be slower. Though I have had a good look at the SQL
generated and it looks pretty good. SQL Server now caches the query plan
for SQL passed through, not just for stored procs anymore.
( See do-stored-procedures-really-boost-performance-in-ms-sql-net for more info)

To mitigate against this I am going to closely observe all SQL being passed to the
db for the first couple of weeks to make sure nothing stupid is going on.

* People need to learn entity framework’s query language. I’d say no big
deal, once you’ve seen a few examples it’s pretty simple.

The big constraint that we had with earlier systems was that the client often mandated “You must use stored procedures”. That mentally is now ten years old, using ORM’s are the way to go and the performance seems up there with stored procs. Last ORM I used was Hibernate with Java and I hated it, but that was 10 years ago. But the Entity framework looks pretty cool. So I say get into it.

Posted in technical | Leave a comment

Parliament Of Australia’s website offline

The Parliament Of Australia‘s website went down yesterday, and it stayed fully down for 36 hours.

According to The brisbane times their solution was typical, just reboot everything:

“It took a little while to track that down and then we had to go through an elaborate rebooting process to actually fully reconnect it to the external world.”

Thompson said the department’s IT staff still “haven’t got to the bottom” of why people outside the Parliament building could not access the site following the upgrade.

The reason why you still “haven’t got to the bottom” of the problem (and I suspect never will), is because you went on an “elaborate rebooting process”. If something goes wrong, you don’t just reboot everything in sight until it works again. You figure out what the problem is first.

But for a site that looks like it’s still running on ASP, has a recommended viewing size of 800×600, and is running LISP over the web (I shudder), and expects semicolons to be treated like the ubiquitous question mark in URLs, where the error page returns a 404, I think they might have bigger problems.

Posted in technical | Leave a comment

Drupal 7, Apache, 256 Meg of Memory and Lighttpd

A few months back I wanted to do a new website. It needed to have a shopping cart, a nice image gallery, a blog, and a wysiwyg way of adding new pages. I decided to go with Drupal 7.

I learnt the ropes, installed it, got my head around nodes, content types, modules, themes, etc etc etc. Got the whole thing to work on my windows 7 box using “php for windows” and “SqLite”. I already had Sql Server installed and didn’t want another DB. And trying to get Drupal to talk to Sql Server didn’t look like fun. (Crazy huh… there goes the whole point of ANSI SQL right).

So two months later after having built the site, I bought a web host, GoDaddy, and deployed it. For the first two days the webserver was pretty much unavailable. On the few occasions when it was actually available, the site was practically unusable. So I cancelled GoDaddy (the first question they ask you: “Are you cancelling because of performance reasons?”) and got my money back.

A week later I went with RackSpace. I bought the smallest server you could find, which was a linux host with 256 meg of memory. Would things work? Well, I deployed it, it worked ok. Converting the site from SqLite to Mysql just didn’t work no matter what I did, but for the moment I’m happy to stick with SqLite.

But… I found that performance still sucked. And boy did it suck! Pages took 5-10 seconds to load, admin pages took 10-15 seconds to load. Not cool! Very annoying considering that Rackspace are meant to be the best… and Drupal is used everywhere… surely things can’t normally be that bad??

I had a bit of a poke around and a few people were mentioning Lighttpd. It seemed to need less memory and performance was meant to be good. After a few hours it was up and running…. and boy, things rock now! Admin pages take 2-3 seconds to load, the website is fast and response, and it can even handle quite a bit of load thanks to my quick and dirty testing with JMeter.

Apparently there’s another light web server out there called NgInx… maybe I’ll give that a go one day, but for the moment I’m happy to stick with Lighttpd.

Oh the site is here: Red Button Furniture.

Posted in technical | Tagged , , | Leave a comment