Mario Jauvin's blog

Jauvin Digital Photography Workflow

2009-04-11T22:17:00.000-07:00

What is a workflow? Basically, it is a series of steps to follow in order to achieve a given result. My digital photography workflow is what I recommend any digital photographer (novice or expert) follow to go from taking the shots all the way to having them in your photo library. Why come up with instructions like that? With digital photography if you do not follow a well thought out recipe you find yourself soon (very soon indeed) with a humongous mess due to the number of shots one can take using this technology. Shots are absolutely free, are they not?

This workflow assumes that you have a good digital camera capable of shooting in raw mode (or see wikipedia on this topic). It also assumes that you have Photoshop to do the post processing. That's it. No hidden clause, no long list. Just those two requirements and the will to learn and use this workflow religiously for all of your digital photography experience. Well, I assume you have Photoshop CS4 but this should mostly work with Photoshop CS3 or CS2. I also must say that this will work either on the Mac or PC version of Photoshop. Thanks to Ben Long for pointing this out to me. I would love to hear your comments and or suggestions using the comment feature at the bottom of the article.

One last note, I made liberal use of screen shots in this article which have all been reduced and centered for the benefit of the reader. Note however that if you click on them you will go to a new window where you can see the original screen shot in all its splendor! All you have to do once you are finished admiring them is to click the back button on your browser to return to the article just where you had left (at least on modern browsers).

So lets start and enjoy.

Access photos from digital camera

My first advice is to never connect your camera to your PC to transfer images. This is both clumsy, slow, often requires use of special camera vendor software with mixed and varied results and uses the battery in your camera for no purposes whatsoever. Instead, invest in a economical memory card reader. These device cost between 20-50 dollars and are well worth their price. I suggest a model that support multiple card format so that you can use it in other contexts (such as memory cards in your phone, other digital cameras). Once you have plugged your memory card into the card reader and connected it to the PC using the USB port, you will get a screen similar to this:

Remember the drive letter where your memory card appeared as (drive D: in my case) as you will use it later.

Initiate transfer of photos to your PC

Then you should start the Adobe Bridge (from your Photoshop installation). Once running, you can select the menu "Get Photos from Camera" as show below:

If you are using Photoshop CS4 as is my case, you also have this little icon in the top left area of the Bridge window:

Select memory card for download

Since this is most likely the first time you have use the photo downloader (this is the program invoked when doing the previous step) from Bridge, you will be using the standard window layout. You should then select the drive corresponding to the memory card that was accessed previously using the drop down menu under "Get Photos from:". In my case it was drive D: as in the picture below:

You will see a dialog box searching for media. Once finished, you should click on the advanced button to view the window with thumbnails for all of the photos on the memory card. This will help you make decisions in the next steps.

Select destination folder

Here is the advanced view of the photo downloader window below. Make sure that you leave all thumbnails selected as we want to always select and process all images on the memory card. I will get back to why we do this later under the advanced options step so for now, please bear with me. You should click on the browse button to locate the folder where you want to copy your images to.

In my case, I selected folder "J:\My Pictures". I would like to make a parenthesis on this very important point. As an amateur photographer and I also believe as a professional one too, it is very important to have two very important attributes for your photo library:

Mobility: you want to be able to look at your pictures wherever you are in different context so that you can show case them in front of a client or simply refer to them when on the road.
Backup: you must always make backup copies of your photos and although I do not yet have this builtin the workflow, I will be making hints to this.

To achieve these attributes (especially the first one) I strongly advise you to acquire a small USB powered external drive (as for example the Western Digital Passport which now comes in 500GB size). I say USB powered because they are much smaller and not having to look for a wall plug is extremely handy especially in a rush or using a laptop. This is what my J: drive is in this workflow.

Destination subfolder settings

You will see under location the mention [6 subfolders] appended to the selected folder. This is due to the default selection of "Shot Date (yyyymmdd)" for the subfolder option and means that with the images in the selected memory devices, 6 different subfolders would be created. Instead I recommend to select the "Custom Name" option from the "Create Subfolder(s):" drop down as below:

Once selected, a text box below will appear with the words "Enter custome name here" already selected:

Destination subfolder name

Do not click in the text box. This will deselect the text and you will have to erase it manually. Simply start typing the name of the subfolder you want to create for this operation. But before you start typing, let me give you my blurb first.

Inside the memory card, I could have pictures from either one shoot or multiple shoots as is the case here. When you have multiple shoots (even on the same day) here is my suggestions on how to proceed. First pick one particular shoot in your list of images. This is where the thumbnails view in the advanced settings is useful. This could be the most important one or the one with the largest number of images but it does not really matter. In my case, I decided that the shoot I selected was the one with pictures of my mother-in-law's birthday party and hence I picked the subfolder name "Cumpleaños de mi suegra" (spanish version of Mother-in-law birthday) and then I prefixed YYYYMM which is the year and two letter month number when the pictures were taken followed by a space so that the full subfolder name is:

"200903 Cumpleaños de mi suegra"

Please note the 0 (zero) prefix in the month number when there is only 1 digit in the month number. This is important for sorting folder names in Windows Explorer or Bridge.

Destination filenames

When copying files from the memory card, names such as IMG_2574.CR2 are meaningless. Therefore, you should select the option "Shot Date (yyyymmdd)" from the "Rename Files:" drop down menu:

Once selected, a text box below the drop down will appear with a number. This number will increment for each image and will be kept and maintained by this instance of Adobe Bridge.

Advanced options

The remaining options should be selected as follow. "Open Adobe Bridge" should be selected and will cause a new windows with Bridge to be opened once all of the processing is completed. "Convert To DNG" should be selected to convert the camera RAW proprietary format (Canon RAW in my case) to the Adobe digital negative format (DNG). This will create smaller files and will store different camera RAW files in one consistent format. Please note that there is no difference in the information stored by a DNG file and the camera RAW file, i.e. the image quality is identical. One major distinction is that metadata is kept within the DNG file as opposed to side XMP files as in camera RAW formats. This makes it easier to handle and move raw files around. "Delete Original Files" should be selected as this will guaranty that once you have processed a file you will never be confused and process it again. This is a new option in Bridge CS4.

When I started in this business of processing images on my memory card, I was trying to be purist and make sure I processed images 1-107 as shoot xyz then images 108-256 as shoot abc etc. Invariably, I would always get confused and go nuts trying to figure out whethere I had or not processed this image and I either missed some files or had duplicates. In order to avoid this I humbly ask all men and women of the earth doing digital photography to adopt the following brain dead and hence, error free, paradigm:

If a file resides on your memory card, it has not been processed
If a file has been processed, it will not be on your memory card

This means that at any time, you pick up your camera, your memory card and off you go. You don't have to worry whether these files on your card have been processed and hence can be erased when you are on site and doing the shoot. You know they have not been processed so if you are out of space, get your backup memory card. We all have one, don't we? I personally have a 16GB main card for all my shootings and a backup 8GB for when I am either really lazy or I am extremely zealous taking pictures.

Metadata option

If you are like me, you are not into spending hours post processing images and in order to save work after, I have some advice. In the "Template to Use:" drop down, I suggest you to select "Basic Metadata". This will create two text boxes, one for "Creator:" and one for "Copyright:". You should put your name in the first and a copyright notice in the next one. To get the © symbol in windows you can press and hold the ALT key, then using the numeric keypad with the Num Lock turned on, type 0169 then release the ALT key. You can find out other codes by typing charmap at the run command. This will bring the character map windows utility.

If you want to add more metadata, then before invoking photo downloader, you should go to the Bridge and select the appropriate command from the Tools menu. Once created, your metadata template will appear in the "Template to Use:" drop down.

This is it, most of the "hard" work is done. This is how your photo downloader window should look like before you press the "Get Photos" button:

And then the usual progress dialog box. One when copying a file and the second when converting the file to DNG format.

If you have CS4 and have, as I sermonized you for the last while, the option to delete the files turned on you should get the following two prompts. Make sure that nothing went wrong during the copy process and then confirm the deletion as below:

Moving images to appropriate folders (optional)

We are almost there. At the beginning, I mentioned that my memory card had several photo shoots on it. You should select the beginning photo of each shoot and then navigate to the ending photo of the shoot and then select Stack/Group as Stack menu option as in the screen shot below:

You should see as many stacks in the Bridge window as there were shoots. The number in the upper left corner of each stack is the number of images included. You should create as many folders (create them inside the subfolder selected earlier as it is far simpler to do it that way) as you have stacks. Remember to use the YYYYMM space naming convention.

You should be left with a screen looking something like that:

What you need to do is click the number in the upper left corner of a stack that you want to process. This will open the stack and select all pictures inside the stack. You should then simply drag any picture from the stack into the corresponding folder which should be in view as in the screen shot below:

And the usual progress dialog box:

Once you have move all of the images into their corresponding folders, you should simply move each folder in their appropriate location and that should be all.

I hope this was instructive and useful.

Performance Visualization

2009-04-11T19:31:00.000-07:00

Just a quick note to tell people that I/we have been running a blog/group site on performance visualization which you can see by clicking here.

If you are not already a member you should see a link on the right side that says "Apply for group membership". Just click on this and you could become a happy member.

CMG Canada in Toronto April 15-16

2008-04-14T05:55:00.000-07:00

Hi, just a note to tell you that I will be presenting the paper "Seeing it all at once with Barry" in Toronto on April 16. This is the visualization paper which Neil and I wrote and presented at CMG in San Diego. I look forward to redoing it with some new visuals and hopefully will report back in a timely manner.

Till later.

CMG 2007 in San Diego

2008-02-10T18:22:00.000-08:00

What happened in San Diego. Well, a lot apart from the fact that I was fortunate enough to have two distinguished speakers speak for the network track, namely Peter Sevcik from Netforecast, a world renowned consultant that knows his stuff and whose opinions are often against the current but proven to be correct after. His pet peeve these days is the application performance index or APDEX (see Apdex Alliance). My good friend Neil Gunther (Neil's blog) and I presented a new visualization for the Apdex index which we believe has a lot of potential. The other distinguished speaker is Raj Jain, a well known academic whose performance book The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling (Hardcover) has been the bible for many old timer performance analyst. Raj current interest is in participating in the design of the Internet 3.0 (I don't know where I was but I don't recall Internet 2.0 but heck, that's just me).

The other really interesting point is that my other good friend Adrian Cockcroft from Netflix (Adrian's blog) was awarded the A.A. Michelson Award which is a great honor and I am very proud for Adrian (see blog entry for details).

Well I could get into why Neil was rolling in laughter on the floor but that would be another full blog entry. All in all, I gave 3 talks, 1 BOFs which had the most attendance in a bof (never seen 30-40 people before). I think the whole conference was a great success. I wished I had reported on it earlier but I am like that, delinquent at updating my blog. Till next post enjoy.

Agile Programming, Continuation

2007-05-12T05:21:00.000-07:00

OK, I said I would continue some other time and if history was any indication, it would have been a long time. But I had the burning urge to say something and my ardent followers were just holding their breath to read it (:-)).

I talked about conventions and ignorance. Here is a case in point (and actually this is a really good example of how to do a multiple selection list in Ruby On Rails).

Here is how to get started. In a windows command prompt window you type:

cd c:\workspaces
rails msexample
cd msexample
ruby script\generate model Movie title:string producer:string
ruby script\generate model Category label:string
ruby script\generate migration CreateCategoriesMovies

Then you edit db\migrate\003_create_categories_movies.rb as follows:


class CreateCategoriesMovies < ActiveRecord::Migration
 def self.up
   create_table :categories_movies, :id => false do |t|
     t.column :category_id, :integer
     t.column :movie_id, :integer
   end
   add_index :categories_movies, [:category_id, :movie_id], :unique => true
 end

 def self.down
   drop_table :categories_movies
 end
end

This will add the has_and_belongs_to_many (habtm) relationship which is describe here. You need to edit the two models also. Here is app\models\category.rb:


class Category < ActiveRecord::Base
 has_and_belongs_to_many :movies
end

Here is app\models\movie.rb:


class Movie < ActiveRecord::Base
 has_and_belongs_to_many :categories
end

We then create the MySQL database using the following commands:


mysqladmin -u root create msexample_development
rake db:migrate

We will then create a set of standard views to be able to edit our list of movies.


ruby script\generate scaffold Movie

We will further customize the form created for this purpose by the scaffold generator by simply adding 2 lines. Here is the app\views\movies\_form.rhtml:


<%= error_messages_for 'movie' %>

<!--[form:movie]-->
<p><label for="movie_title">Title</label><br/>
<%= text_field 'movie', 'title'  %></p>

<p><label for="movie_producer">Producer</label><br/>
<%= text_field 'movie', 'producer'  %></p>

<p><label for="movie_categories">Categories</label><br/>
<%= select('movie', 'category_ids', Category.find(:all).collect {|p| [ p.label, p.id ] }, { :include_blank => true}, {:multiple => true, :size => 3})  %></p>
<!--[eoform:movie]-->

The last thing we need to do is to create the categories scaffold (the rails thing that will allow us to edit categories).


ruby script\generate controller Categories

We then edit app\controllers\categories_controller.rb:


class CategoriesController < ApplicationController
 scaffold :category
end

We are now ready to start the server and navigate to http://localhost:3000/categories to edit and add a few categories and then http://localhost:3000/movies to edit our movies and assign categories to them.


ruby script\server

In my next post I will describe the huge number of conventions that went into making this possible. And it is worthy to note that this only required 60 lines of code (as the rake stats command show below will tell you) of which 49 were created by the rails generation software:


C:\Documents and Settings\mario\workspaces\radrails2\msexample>rake stats
(in C:/Documents and Settings/mario/workspaces/radrails2/msexample)
+----------------------+-------+-------+---------+---------+-----+-------+
| Name                 | Lines |   LOC | Classes | Methods | M/C | LOC/M |
+----------------------+-------+-------+---------+---------+-----+-------+
| Controllers          |    61 |    48 |       3 |       8 |   2 |     4 |
| Helpers              |     7 |     6 |       0 |       0 |   0 |     0 |
| Models               |     6 |     6 |       2 |       0 |   0 |     0 |
| Libraries            |     0 |     0 |       0 |       0 |   0 |     0 |
| Components           |     0 |     0 |       0 |       0 |   0 |     0 |
| Integration tests    |     0 |     0 |       0 |       0 |   0 |     0 |
| Functional tests     |   110 |    79 |       4 |      13 |   3 |     4 |
| Unit tests           |    20 |    14 |       2 |       2 |   1 |     5 |
+----------------------+-------+-------+---------+---------+-----+-------+
| Total                |   204 |   153 |      11 |      23 |   2 |     4 |
+----------------------+-------+-------+---------+---------+-----+-------+
  Code LOC: 60     Test LOC: 93     Code to Test Ratio: 1:1.6

Agile Programming, What's That?

2007-05-11T10:16:00.000-07:00

Ok, it has been long since my last post (as usual) and I thought I would talk about a new different topic.

I am currently involved working on a web application and I was exposed to a concept of agile programming about a year ago (maybe more, time flies, does it not) and I looked at it and said "Oh yeah! Just a new fad that will go away" and I continued slugging away on my J2EE application (stuff for real programmers, right?). Well it did not go away, quite the contrary and I am now using it.

Just to show you should keep your mouth shut or (as Jean-Jacques Rousseau said: put your work one thousand times on the mill - sound much better in french) I should have thought 1,000 times before saying something. But actually, you did not come here to read my ramblings on my lack of good judgment (or maybe you did, in which case I can suggest a couple really good therapist) so what is this agile programming.

Well, pushed to the limit, which in reality is never attained, it's basically having your client next to you while you develop your application and have him/her give you immediate and continuous feedback. Now, interesting as it may be, this approach is a recipe for never ending projects because giving your client the opportunity to change his mind 100 times is not necessarily productive. Also, unless you are on a time and material contract basis (or employee) it is simply not cost feasible on a fixed price basis.

Although this extreme approach is never used in the real business world (although you could read a good article on O'Reilly Onlamp web site) , you can take a less drastic version and actually invite your client at some crucial points in the application and have him or her give you feedback. The agile programming approach is usually most effective at the very beginning stage of the development when you quickly and effectively demonstrate the initial data abstractions and how to connect them to a web page. You can quickly show how to perform typical create/read/update/delete (or CRUD) operation and this with your client present and giving you feedback.

So far, I have not mentioned the word Ruby nor the word Rails. This is actually the tool I used (and there are lots other but I would say none as popular). Ruby is an interpreted object oriented programming language with loads of really nice features. Rails is a web application framework (like STRUTS/SPRING/JSF etc... on Java) written in the ruby language.

Getting back to the agility effect, ruby on rails (or RoR as it is affectionately referred to by hardened converts which I am not :-) ) (pardon my poetic license with the two closing parenthesis before and after :-)) is a very good tool for implementing an agile programming paradigm. It does this by using convention rather than configuration to do most things. This means that if you follow all of the conventions, you will be able to write very little code to do most things. Now, this is where the agile paradigm fails when you need to deviate from the convention (either by necessity or by ignorance). The necessity may come because you have existing conventions that you must adhere to or that the business problem you have requires it. You may do things without using the RoR (now, does that make me a liar and turn me into a hardened convert?) conventions simply due to ignorance and lets face it, a true scalable business class production web application is not a simple thing and any framework that must support any kind of such application is even more so complex by nature and RoR is such a beast. So ignoring one of the hundreds of conventions in RoR is not a sin to be terribly ashamed of.

Since I do not have more time I will leave this post as is but I will resume this discussion later.

Performance Visualization - What the heck?

2007-02-15T15:18:00.000-08:00

Yeah, exactly what I asked the first time I started thinking about it. But then I met my buddy Neil at CMG (Computer Measurement Group) in Reno (brrr, was it cold there) and we started talking about hot topics and performance visualization came up. Actually, I don't recall if this happened before or after we had a few beers. This is not that important because most really good ideas came after a few beers.

So, back to the topic at hand. Visualization is not a new concept. It has been used in several fields. Here is a list of web site with various applications of visualization:

Here is a technical article on visualization as it applies to computer and network performance.

Performance Visualization May 2005 Article by Neil

Now the big complaint from my good friend Neil is that people in the performance field are content with the usual visualization techniques such as boring bar charts, plots and pie charts simply because they are not aware that other "better" (I will come back later with what better means) visualization techniques are possible. Performance tool vendors are not interested in funding R&D for new performance visualization techniques for the same reason and also quite simply because no one is asking for it.

The purpose of this entry is to talk about the problem with the intent of gathering interest in this very interesting problem which is truly not a simple one to solve.

The problem with visualization is that one needs to come up with a way to represent a digital abstraction (performance data) in a way which is natural or easy for the human computer (the brain) to interpret. Now, put this simply this sounds like a rather simple proposition. However, it is actually extremely difficult to solve. In this discussion, if a given visualization technique is easy for the human brain to interpret it is said to have a good impedance match. The reverse condition is said to have a poor impedance match.

The reason this problem is not an easy one to solve is because the scientific rules for defining what is a good impedance match have not been well defined or studied. Several techniques have been tagged as having a good impedance match but rather on subjective terms using words like "cool", "neat" and even the odd "wow". This does little though for providing a methodology for coming up with visualization techniques of performance data that have a scientifically derived good impedance match.

The other problem with visualization in general is that quite often a good visualization is one within a three dimensional universe. This is probably the case because the human brain is accustomed to operate in such an environment. However, representing a 3D visualization in a 2D computer screen is a difficult, costly and challenging approach. This problem has become much more accessible to the masses with the advent of super computers like dual-core CPUs, hyper threading cores, etc and the great advancements in video card technology in the last 5 years.

The problem domain where visualization has gather momentum is in the field of marketing. This is because a great deal of visualization customization (as opposed to generalization for performance data) is possible from a cost point of view because of the nature of the problem. 1 particular visualization is applicable to 1 and only 1 product or service.

In my next entry, I will try to expand on examples in some or all of the fields discussed so far. As always, your comments are welcomed.

At long last, I have a blog

2006-12-19T11:55:00.000-08:00

I had been meaning for too long and now I have finally done it. Here is my blog. In this blog you will find topics of interest to computer and network performance, software development especially on Java and stuff related to other interest of mine such as VOIP, open source project, network protocols etc. I truly hope you enjoy your visit.