One-off programming

We needed to send some data to a partner for an analysis. They need the files in .csv format and some other draconian specifications. To get the data, I had to run a set of SQL statements that resulted in 4 files, and I had to do this for 5 different companies, which resulted in 20 files.

However, our partner company needed all the data in 4 files only. So I needed to take 5 files and combine them all into 1. In addition, I had to add the store number as the first column in the CSV on every record in every CSV.

Sounds like a job for Excel, right? Well, I've never had luck making Excel understand CSV files (along with all of the double quote and comma characters) and when I opend the files in Ultra Edit, it completely fell down on itself. These are BIG files.

Enter Stage Left: C#. I fired up VS 2010 and wrote 132 lines including curly braces, empty lines, comments, using statements, etc. This app first takes all 5 store files for a certain query, adds "MerchantID," to the first line in the file, then appends on a string such as "44000," to every line and saves it out to a new file. Step 1 complete. Now I have 20 files with the Company Identifier as the first column in each.

Step 2, read in all 5 files for each query, use the first line from the first file, and then just append on all of the data lines from all of those files, into 1 giant aggregate file. Do this for all 4 of the query types.

Now I was left with just 4 files, all companies combined together and identified on every record, and the original data was still just as pristine as it ever was (meaning, I didn't have to worry about Excel "helping" me with doubling the double quote character, adding extra commas, etc.)

When I ran the app, it took about 12 seconds to process all of that data. Ultraedit couldn't handle it (in the way that I wanted to use it), and Excel even struggled with the largest file (lots of waiting while Excel played with memory).

This is one of the benefits to being a programmer. If you want to make your life easier while you learn quite a bit about how your machine actually works under the covers, do yourself a favor and start writing programs to deal with text files. You'll learn about how string manipulation affects your processor and memory, about how file IO works, how streams work (in C# at least), and you'll also build yourself a few tools to do file manipulation that you will invaluable in very many unplanned situations.

MVC and TDC: Starting Out

I just started a new project and chose the standard MVC3 template. I left the account stuff in there for now, and also the Home stuff too.

I then added an ActionResult method called SystemStatus. I altered the Shared Layout view (deleted some stuff, added in SystemStatus). I played with the ViewBad.Message and some other properties to make sure my changes were taking effect.

I added a SystemStatus View by right clicking on the method in my controller and choosing "Add View". I changed the name but left the defaults.

I then added a Tools Controller and a couple of views for that one, too. Altered the Shared Layout View to give an option for my Tools stuff. All is working.

As for unit tests, I just kind of copied the existing unit tests that were built (I said "Yes" to the prompt asking if I wanted an associated Tests project.) So, my code is covered with tests. Kind of.

Now, I need to get the Model hooked up to an existing database. I'm using TDC as the backend, and just bolting on a quick reporting/small maintenance type app to it.

So, after downloading Castle, NHibernate, Entity Framework 4, etc, I settled on EF4 for the time being.

To get this up and going, I right clicked my Model folder and chose Add New Item, ADO.NET Entity Data Model. From there, I had to feel around to connect the DB, then to choose the tables I wanted. In the future, to edit this, you have to choose to open the .edmx file, not the file under it.

It then generated a bunch of stuff (hidden) that allowed me to find objects like tbl_MarketingCode (not my name).

From the controller, I generate a list of those, and pass that as the first argument to View(). I had to goto the view itself and add in, at the top, a @model List line. Can we only pass 1 thing to a view? To get to this variable that was passed, you just reference it in the view with the Model keyword. What if it needs other things?

What So Many Web Programmers Don't Seem to Understand...

...is that all they are doing is playing with strings.

The browser sends you a string. Everything you do with that string is merely to create another string to send back to the client.

That's all we do.

Doubt me? Then you are an idiot. And if you think for a second that any user cares if you are using MVC, Ruby, PHP, .Net (C# or VB or any other flavor), C, D, F#, Haskell, whatever, then you are even more of an idiot.

Your job is to take a request string. Maybe your language splits it out into a nice neat object for you, maybe not. Learn what a request string looks like. Learn to split it up on your own. I did this in VB years ago. I learned a LOT.

Once you figure out that the only thing you know about any page run in your site is what gets sent to you, then you start to understand just how completely insecure your app is. There are some very easy ways that a script kiddie can let the browser generate a real request, stop that in its tracks before it actually gets sent, and alter any data in it that he wants.

What does this mean to you? It means that no matter how much touchy feely javascript/validator methods you have in the client, you can't trust anything you receive in a request object. You must ALWAYS verify data content, field lengths, formats yourself. You must ALWAYS determine if the cookie that tells you that userA is logged in is valid and makes sense. You must ALWAYS determine if userA has rights to edit the object that the request claims to want to edit.

Real world example. Orders on a site are numbered sequentially. It happens. userA has placed order5. userA is a hacker, and tries to edit the shipping address of a previously placed order, order3. You MUST check to see if userA has rights to edit order3.

No MVC, framework, library, whatever, will ever be able to do that work for you.

So, back to my original point, all you do is play with strings.

What does this mean to you? Get off your high horse about what language you are using. All that we should ever care about is what language is faster for us, and still gives us the flexibility to control, to a minute detail if needed, what string gets return back to the client.

jQuery AJAX with .Net, and No Dependencies

Goal: to have an xcopy deployable .Net 2.0 project, using AJAX and JSON and jQuery. This solution requires no web.config changes, no other binaries, no GAC stuff of any kind.

Create a page, nodep.aspx. Add jquery.js to a js folder in your project (get jquery from the net). Add an include to this file in nodep.aspx.

Add a Generic Handler file to the project, called whnodep.ashx.

Add this to nodep.aspx, anywhere you want, really.



Add this to the body




In whnodep.ashx, put this code in the ProcessRequest method, take out all code and put in this:

if (context.Request["pageAction"] == "DoSearch")
DoSearch(context);

Add this procedure:


public void DoSearch(HttpContext context)
{
string result = "";
string sSearch = "";

sSearch = context.Request["searchterm"];

result = "{ \"letterarray\": [";
for (int i = 0; i < sSearch.Length; i++ )
{
result += "{ \"letter\": \"" + sSearch.Substring(i, 1) + "\" }";
if (i < sSearch.Length - 1)
result += ", ";
}
result += "] } ";
context.Response.Write(result);
}

Done. Should work flawlessly.

JSON -- headache eliminator, part 1

I haven't used JSON very much. I like it, but I just haven't had a chance to really get down into it too deeply. However, this is the 2nd time that I've been bitten by a stupid syntax thing. The solution is on many many blogs out there, but hopefully having typed this myself, I might finally remember it.

Server side, when you are creating your object, you might concentrate on the data, and end up sending a string like this:

{ "someproperty" : "10" }

And, client side, you want to use it like this:

var o = eval(result);

Kaboom! What the hell? Why?

Because you forgot the parentheses:

var o = eval( "(" + result + ")" );

All is right with the world again.

DO NOT FORGET THE DAMN PAREN WRAPPERS!!!!

Navigation Controller Fun

Add 2 (plus 2 headers automatically added) to your Classes group.  Nav1ViewController and Nav2ViewController, both using the template of UIViewController subclass.

Drag a Navigation Controller over from the toolbox and drop it onto the Tab Bar Controller.  Take a Tab Bar Item and do the same thing. Run it.  I think it "just works" and shows a new view with a navigation controller on the third tab.  

Add 2 new Nibs, from View Nib, called NavView1 and NavView2.  Set their File's Owner's class appropriately.  Associate the View Outlet with the view.  Run it.

Now, add a button to the NavView1.xib.  Go to your Nav1ViewController.h file.  Put in this line: 

-(IBAction)LoadNextPane:(id)sender; after the end of he Interface block, before the @end block.

Put this code in the .m file:

-(IBAction)LoadNextPane:(id)sender{

// Do something

UIViewController *targetViewController;

targetViewController = [[SecondViewController alloc] initWithNibName:@"NavView2" bundle:nil];

[[self navigationController] pushViewController:targetViewController animated:YES];

}

At the top of the .m file, you must include an import to the file that tells the compiler about what SecondViewController really is.

#import "SecondViewController.h".

Now we have an action (I think of it as an event, but it probably isn't).  Go back to the NavView1.xib file.

Associate your button's "Touch Up Inside" to the File's Owner, and choose the "LoadNextPane" option.  It's the only one that should appear.  Run it.  Kaboom.  Not sure why right now.

The problem was that I had not, in MainWindow.xib set the Nav1 View Controller (Navigation Item) 's class to Nav1ViewController.  Once I did that, the 2nd nav view scrolled on screen with the button push.

iPhone -- Interface Builder, Part 2

See iPhone -- Interface Builder, Part 1 for instructions to get to this point.

So, let's make SecondView use a code file.  Right click on Classes and choose "New File".  On the left of the New File window, under iPhone OS, choose Cocoa Touch Class, and on the right, choose UIVIewController subclass.  Click Next, give it a name of SecondViewController.m and his Finish.  It made the .h for you, automatically.  Build and Run still works, but we really haven't done anything.

Open your SecondView.XIB and change its class to SecondViewController.  It still builds and runs.  Okay...so what have we allowed ourselves to do?  Well, I THINK that we can now override things on the second view, respond to events, set items on it, etc.  So, we have a code file instead of just a dumb view with text on it.

Now, let's get FirstView.xib built.  Right click on "Resources", add new file, choose User Interfaces, and choose View XIB.  A part of me really really really wanted to be able to choose "View Controller XIB" but alas, that is not an option.  Give it a name of FirstView.xib.  Change the File's Owner class to FirstViewController.  Set the File's Owner's "view" outlet to the view that was created.  Add a label to the view so we know it is OUR view, and not the default one they made.  Go back to MainWindow.xib, and set the "Selected First View Control" to load from NIB FirstView.  Finally, delete the View tree under "Selected First View Controller".

Build and run, and you should see YOUR first view already loaded.

Also, go back to MainWindow.xib and change the "Second View Controller"'s class to "SecondViewController".  Not sure if that makes a difference, but I think it does.

iPhone -- Interface Builder, Part 1

Interface Builder is currently the bane of my existence.  I just don't quite understand what I'm doing, and I definitely don't understand HOW it is working.  I'll outline the latest project here, and how I sort of got it working.  I mean, I DID get it working, but I'm not sure about some of the decisions I made and how it will bite me down the road.

I started out by creating a Tab Bar Application based project in Xcode.  I ran it, and it compiled.  Check.  The MainWindow.xib file already has the first view loaded into it, but the 2nd is done via magic.  I wanted to figure out what the magic really was.  The "First View" has this text in a label:  "Optionally move this view into a separate nib file and set the nib name in the First View Controller Attributes."  Well, why not try this?  Let's see if I can replicate this as I type, mistakes and all.

First, I looked at the inspector for the MainWindow.xib file.  File Owner is a class of "UIApplication".  It also has an entry called "App Delegate", and it's class is "TBNB001AppDelegate"  This is the name of my application (TBNB001), and when I made the project, Xcode created TBNB001AppDelegate.h and .m for me automatically.  This is what is assigned to the "App Delegate".  

MianWindow.xib has a Window, but that is pretty uninteresting, I think, because I believe that all iPhone apps need to have 1 window, and that's all.  It's a class of UIWindow, so it hasn't been altered or overridden, etc.

Then we get to the last parent level object, "Tab Bar Controller".  It's type is UITabBarController, but it isn't listed in the Inspector.  That's...odd.  Why?  I mean, I don't plan on overriding or catching any events from the Tab Bar (actually, I do later on, but that is my project specific, and I'll deal with that later).  But shouldn't it have a class of UITabBarController, too?

Under "Tab Bar Controller" is "Tab Bar".  Okay...same thing as the Controller above...it has a type, but no specified Class.  I HAVE to assume here that if the Class is blank, then it is just the same as putting in the default class.

Next under "Tab Bar Controller" is "Selected First View Controller".  Not sure if "Selected" is set from some property I couldn't find, or if it was just called that.  No idea.  It's class is called FirstViewController -- again, this relates to 2 files of the same name + .h and .m that Xcode created for me automatically.  The View Controller has a connection to the view inside of it.  A View Controller must have a default Outlet defined for a view, cause it sure isn't anywhere in the code for "FirstViewController.h" or .m.

Now, look at the structure for "View Controller (Second)"  It's class and type are both UIViewController.  VC1 has a view outlet that is used, but VC2's view is unassigned.  On the last page of the inspector, VC1 has no class outlets defined, but VC2 does, and it is a UIView.

I'm pretty confused, I have to say.

Finally, though, VC2 is set to load from a NIB called "SecondView".  Ahh, that magic part, I guess.  So, I do have another file, and it is called SecondView.xib.  I don't have one for the first, but I do for the second.  The first is defined in place in the MainWindow.xib file, under the Tab Bar Controller -> Selected First View Controller structure.  The Second View Controller doesn't have that, because it gets loaded from an external .NIB.

Now, for "SecondView.xib".  It's File Owner's class is UIViewController.  It has an outlet defined, and it is associated to the View in the file.  All of this was done for us already. 

SecondView doesn't have a set of associated code files.  Well, I really like to do things in code because it usually ends up being more flexible later on, plus I find it helps me figure out WHY things work the way they do.