Software Estimation that Works, Part 1: Ranges. Confidence and Risk

Estimates Cannot Be Precise

“How long will that take?” No other question stops a tech team dead in it’s tracks.

Without exaggeration, it is like asking a new parent, “So how much do you think it will cost to send you kid to college?” If you are that parent and paid your own way, or maybe even paid for a previous kid; despite that experience, you know that you are hazarding a guess. It might be an informed guess, but it still fraught with uncertainty. Worse yet, the question implies a bunch of new worries heading your way. They are not asking some 3rd party financial advisor. They are asking YOU, the person who is expected to be able to pay this hypothetical bill once it is less hypothetical.

When you ask a tech team to estimate a date, they are just as aware that 1) they are hazarding a guess and 2) they will be expected to do the work to make that guess a reality.

People hate to be put on the spot, so there is some jujitsu in this method of estimation. Instead of asking for specific estimate, ask for a range. Giving people the wiggle room gets them out of the “I have to be exact, because I am going to be held to it” mentality. Be aware that phrases like “how long” or “when do you think you can be done” are being heard as demands for commitments.

Asking estimators for a likely range instead of a date has several benefits beyond putting your estimators at ease:

* gets everyone beyond the illusion of precise estimates
* gives you an idea of the team’s confidence in their estimates
* shows you where the risks are in the work ahead

High, Medium, Low

My favorite range estimation method is called High/Medium/Low or the 1-1-4 method. Ask your estimators to give you a range in three numbers:
* a low end estimate, if everything goes right
* a most likely estimate, if things go as you expect
* a high end estimate, if things go badly.

If you estimate your range in these three parts, you are thinking about the task from an optimistic, pessimistic, and a realistic point of view. Most teams sway one way or the other: either delivering overly optimistic or pessimistic estimates. Consciously trying each mode exposes more data. In the optimistic mode the team is describing what the work would look like if everything went right. In the pessimistic mode, they are forced to think through possible risks, complications, or just time sucks. Using experience to see how these two extremes balance out, the team usually ends on a more informed realistic estimate than if they just did a single point.

As a person gathering this data, you are now exposed to more raw data. You are not getting estimates that may be padding, or may be waiting for your padding. You have a High, Median, and Low to work with.

Working with Ranges

If you assume that low, median, and high end fall on some sort of bell curve, you combine them with:

This is where the name 1-1-4 comes from. You multiply the MEDIAN, or the most likely scenario by 4, and add in the best and worst case. Then you divide the total by 6. This gives you a weighted average that assumes that likely estimates are 4 times more likely than the worst or best cases. This method allows you to:

* factor in large deviations caused by worst case scenarios
* spot if your estimators best cases and most likely are often the same (too optimistic)

Example

Task: Integrate with a 3rd party data service.

LOW	1d	The system probably uses JSON or another well known format. It could be fetched by a simple URL request and parsed with our existing systems
MEDIAN	3d	The system probably uses a well known format, but their will probably need to be some tweaks to the serialization library. If they are not using standard HTTP retry and error codes, we might have to add some custom handlers and additional unit tests.
HIGH	15d	They are using a custom serialization, so we will have to implement our own parser/serializers. If they did that, we probably also have odd-ball network issues. They could be using SSH security and require some sort of certificate setup.

AVERAGE = (4*MEDIAN + LOW + HIGH)/6

AVERAGE = (4*3 + 1 + 15)/6 = 4.7d

AVERAGE rounded to days: 5 days

Confidence

In the above example, there is a wide swing between the HIGH(15d), MEDIAN(3d), and LOW(1d). When you are looking at a set of estimates that you are using to build up a project schedule, you want to look at the confidence your team has in their estimates and the risks behind these estimates.

Confidence is how wide is the range. The more sure a team is of an estimate, the narrower the range.

CONFIDENCE = MEDIAN/AVERAGE

CONFIDENCE = 3/5 = 60%

The closer the confidence percentage is to 100%, then there narrower the range. If you are looking through a set of estimates, you want to examine the ones with low confidence percentages. The team might not be able to give a high confident estimates for lots of reasons. Diving into open questions might help gain confidence and narrow the gap.

Do the requirements need clarification? Does some technology need to be tested or researched? Are you dealing with a un-tested or unreliable vendor? Asking these questions might help you add additional data to the estimate to increase confidence. Or, it could be early detection of a problem with the plan.

Risk

If you have two estimates with low confidence percentages, they might not warrant the same attention:

TASK	LOW	MEDIAN	HIGH	AVERAGE	CONFIDENCE	RISK
Big Task	2	4	25	7.2 (8 whole days)	55%	17 Days
Small Task	1	1	5	1.7 (2 whole days)	59%	3 Days

RISK = HIGH – AVERAGE

Both the big task and the small task have about the same low confidence level, meaning that the team sees that there is a wide variance in their possible outcomes. However, if the big task goes bad it could add 17 days to your workload, while if the small task goes bad it will only add 3.

The Confidence is a relative measure, the risk is an absolute size.

When you spot big risks, it is usually because you have a large task. The best strategy is to work with the team to see if you can break the task down into smaller tasks and identify if the risk is the culmination of a lot of small risks that will all need to be managed, or is there one really risky element throwing everything off, which could be tackled up front.

In asking your team to think about things in an range driven by optimistic, pessimistic, and realistic points of view; you have asked them to dive into each estimate more than just giving you a date. You have also give them a vehicle to quantify confidence and risks in the estimations which give you and team essential data for developing schedules and plans.

How Software Estimates Fail Part 4: Lots of Little Estimates

Last week we talked about how easily estimates go wrong when you try to make giant ones up front, but they can also fail easily when you try to add up lots of small ones.

Once again, it comes down to simple math and how optimistic we are about our planning. If I were to ask you to estimate your grocery bill, it would probably turn out something like:

Item	Estimated Cost	Actual Cost	Difference
Milk	$2.99	$3.99	-$1.00
Bread	$3.99	$3.59	+$.40
Cheese	$7.99	$6.59	+$1.40
Tomato	$0.99	$1.19	-0.20
Lettuce	$3.49	$4.99	-$1.50
TOTAL	$19.45	$20.35	-$0.9

In this example, we are ninety cents over budget, which is only a (0.90/20.35) 4.4% budget overage. In software projects, running 4.4% over budget would be a victory!

Groceries are the real world, and estimates work differently there. For one thing, people have lots of experience in the real world and have an intuitive feel for the data. People tend to be more realistic than optimistic. In the above example, none of the estimates are right, but the over-estimates and the under-estimates tend to even each other out. The end result is a very accurate estimate.

In the 10th Annual report from the Standish Group, only 4% of developers over estimated their schedules. These results may be up for debate, but I would say that the trend is correct. Developers underestimate much more than than the over-estimate. This could be due to many reasons, including:

• pressure to create “acceptable” estimates
• lack of experience
• lack of expertise in the area to be estimated
• incomplete requirements or specifications
• missing details
• forgotten work

So unlike groceries, software has an overwhelming trend of underestimation. When all the estimates are added up, instead of evening out, they compound each other into one giant underestimate.

We have all dealt with the Grand Estimator, who hears a brief sketch of the project and declares “It can be done by X date!” We cringe because we know that 1) they are probably wrong, 2) they are wrong because they have no details behind their estimate, and 3) because we are usually the ones left with the ball after they make up a date and run of to their next project.

To escape that insanity, we try to use data to back up our dates. We have all seen it, and some of us have even done it. Many of us are guilty of the death by many small estimate. We list out all the work to be done, give each one an estimate, and total them. All this does if give you a false sense of security. Unlike the grocery store, the totals usually don’t work much better than the Grand Estimator.

How Software Estimates Fail Part 3: The Grand Estimate

Classic project planning work backwards, you pick a large goal that will propel the business forward and add the right mix of enough man power and/or time to hit your goals. Make millions. Retire.

If it were that simple, none of us would be dialing into 9am scrum meetings.There is a reason why when people start on version 2, they always lament if they had only known at the beginning of version 1 what they know now. One goal of realistic estimation should be to get you to version 2 as quickly as possible. Don’t waste a lot of time or resources estimating, planning, and attempting version one.

Business will howl that you are essentially making version 1 throw away. They are right for the point of view of getting to the first date possible, but they are wrong from the point of view of the first date for the first plausible product.

If we remember The Cone of Uncertainty, estimations at the beginning of a project inaccurate by a factor of 4. The bigger your project, the bigger your cone, the bigger the time you can be off. Scrum isn’t able to magically reduce the cone of uncertainty. It uses a series of smaller cones. If someone does the math, you will see that the estimates of 5 subsequent 1 year long projects have the same error rate of 1 single 5 year project.

	Low	High	x # of Iterations	Total Range
5 yr plan	2.5 yrs	2.5 yrs	x1	2.5 – 10 yrs
1 yr plan	0.5 yrs	2 yrs	x5	2.5 – 10 yrs

So what’s the different?

In the second case, you didn’t have to wait as long to get to version 2. Things you learned along the way can be applied more quickly. Changing market conditions can be taken into account. If you are off on a 1 year plan, you could end up being 1 year over schedule. In a five year plan, you could be 5 years late! Not many business can stomach that kind of risk.

Let’s look at a real world example. Around 2000, both Apple and Microsoft decided to re-invent their flagship operating systems.

MicroSoft started the project in 2000, planning for the first release in 2005. By 2007, they realized that they needed to launch something to the market, so they cut a version called “Vista,” which had dropped the much anticipated new file system and display technology. The big new feature was the security model. The planned version made it to market in 2009, 4 years late.

In terms of the Cone of Uncertainty, MicroSoft beat the odds, they had better than market estimations and hit a date that most organizations would never be able to replicate.

In the same timeframe, Apple launched OS X and six incremental versions. No single version was a revolution, which was reflected by their lower price points. But each version was a market success where MicroSoft has suffered from poor adoption. Many of the features that MicroSoft user had to wait for until 2007 or 2009, were in the market years before for Apple users. The “big” upgrades scared many MicroSoft users, particularly conservative IT departments. Apple OS updates were seen more like routine patches, just ones that cost some money.

Both companies had a similar problem and were up against similar estimations and project management risks. Apple decided to ship often and early while MicroSoft bet big. Sometimes the big bet pays off big, but it is the exception and not the rule.

How Software Estimates Fail Part 2: Estimating Once

This is the “Cone of Uncertainty.” The numbers behind the cone are based upon surveys of thousands of software projects and the differences between their estimated and actual schedules. As you travel into the cone, you are travelling further into the project’s timeline. Estimates made later in a project are base upon more data and more accurate. So far, all of this is common sense.

Two things most people don’t understand about the cone:

• It’s a best case scenario
• It expects the teams to re-evaluate estimates at each step

The cone represents the BEST CASE SCENARIO, you live OUTSIDE of the cone. At a project’s initial concept, the best estimator should expect to be off by AT LEAST a factor of 4.

Second, the estimates do no magically get more accurate. This is where you get to the “Wormhole of Uncertainty,” where you find a whole other project hiding in the last 10% of your current project.

It is this second point where most teams miss the point. The cone narrows because at each phase the teams take what they have learned and re-evaluate the estimate.  That means that previous estimates are off the table in doing the next estimate. A team may have initially estimated 1 month, and come back in a later phase estimating 5 months.

Most managers would howl that this estimate has increased from the original, so it is rejected. This is how you guarantee a continued stream of bad estimates. A more realistic project would recheck their estimates at each phase so that they can provide a continued stream of the most accurate data. At each phase, the accuracy of the estimates should increase:

	Estimate	Factor	Low End	High end
Initial Concept	1 month	0.25x – 4.00x	1 wk	4 months
Approved Product Definition	2 month	0.50x – 2.00x	1 month	2 month
Complete Requirements	10 wks	0.67x – 1.50x	7wks	15 wks
UI Design Complete	11 wks	0.80x – 0.125x	9 wks	14 wks
Detailed Design	11 wks	0.94x – 1.19x	10 wks	13 wks

As a developer, expect someone to bust out the cone at sometime. Usually they do this to claim some level of accuracy, or limit on how much “padding” they will accept. The cone is the exact opposite. The cone serve to tell you the upper limit of your accuracy, not the lower limit.

Most importantly, the model states that projects get better and better estimates as they go on. This means that you HAVE to keep redoing estimates. That does not mean “validating” previous estimates. It mean using all of your new information to get a new estimate.

How Software Estimates Fails Part 1: What Is an Estimate Anyways?

Nothing dooms a project faster than its first estimate. From the fateful moment that someone even hints at one; the launch date, budget and staffing all seem to be broadcast to every possible stakeholder and locked down in an iron clad contract. This kind of behavior is why so many gun-shy developers just stopped giving estimates all together and refuse to do anything other than scrum. “We’ll get there when we get there,” they all cheer.

There are plenty of ways to go wrong making estimations. The first and worst mistake that I see over and over again is simply not knowing what an estimate is, which is why simply communicating them usually turns so disastrous.

Let’s start with what estimates are NOT:

• The enemy
• A schedule
• A commitment
• A goal
• A decision

Most people who work in software understand the Project Management Triangle:

The project schedule cannot be defined as something that stands on its own. Scope, budget, and schedule must all be balanced with each other. If you bring in the schedule, you must bring in the scope and or expand the budget. In a “perfect” project, the team would be able to adjust all three as needed. In most cases, schedule and budget are “pinned” so all you can do is keep cutting scope to keep on track.

There is the underlying assumption that quality is not up for grabs. Dishonest or wishful thinking often result in scope, schedule, and budget all staying “on target” while the loss of quality is left to the customers to find out for themselves. Those sort of teams don’t usually stay in business for long.

Estimations are part of a similar triangle:

Estimations are defined by their relationship to a target and a commitment. Unlike the Project Management Triangle, estimates are ALWAYS pinned, meaning that they are not up for negotiation.

A Target is the goal that you want to hit. It is the combination of scope, budget and schedule.The business can adjust the target, but the facts of business might limit what you can do. For example, setting a target of entering the Tablet market 3 years after Apple’s iPad might not be as valuable as beating them to it.

An Estimate is just data. It is the collective analysis and guesswork of your team of experts to best understand the variables involved in hitting the Target. It is important to always think of an estimate simply data, or a measurement. You don’t negotiate with thermometers when it is too hot, and you shouldn’t negotiate with estimates.

A Commitment is where an estimate and a Target come together. Depending upon the risk in the estimate, or in missing the target, you adjust your commitment. You negotiate a commitment. If you work for the most laid back company in the world and they don’t mind due dates whizzing past with nothing to show, then there is little reason to worry. Just set your commitment to the same date as your estimate and relax. Of course, those kinds of companies don’t tend to stay in business long either.

If you are working on a fixed spend bid, and each dollar over the proposed budget, you swallow; then the amount of “safety” you put in your commitment is a lot more important.

Simply getting teams to understand the difference between estimates, targets, and commitments will alleviate a lot of stress and let people talk about estimates without fear that they are being translated into iron clad promises. The more people you can get involved in critically looking at estimates, usually the better your data is. If the team is afraid to communicate honest estimates up front, then your chances of success are just that much slimmer.

Quilting, Because Sewing isn’t OCD Enough

After a couple years of random and smallish sewing projects, I wanted to tackle something bigger. Okay, to be honest, my best friend did. She talked me into signing up for a quilting class with her. Sam is a competitive girl, so I should have know that she wan’t just looking for moral support. She was looking for a whipping boy. Some guys never learn.

It seemed like a good deal. Five Sundays in a row, from 2-4:30. For those scant hours a week, you leave with a heirloom quilt of your own design. I was reading the class listing already imaging laying ontop a bright, cotton quilt during a cool spring morning, reading books and sipping tea. I could practically feel the cuddle of the comforting fabric against me and my little dog as we snuggle into an evening of video game violence. In my mind, I was five weeks from now with all the work behind me. It seemed all so straight forward.

Week 1: False Confidence

After signing over checks and our next 5 Sundays to the Sewing Arts Center, we were ushered to the classroom to meet our instructor, Susan. Her soothing Danish accent and confident smile made everything sound so plausible. She handed out instruction sheets and explained the pattern and fabrics that we should choose. It was when she pulled out a finished sample that I started to understand what I signed up for.

Sam and I headed to the LA Fashion District to buy fabrics. I pride myself on being able to get a bunch of odd/loud patterns that somehow just work. I had my armfuls done in 30 minutes and was ready to go. Sam spent an hour more hemming and hawing. My load was full of owls and bees. I was feeling very on trend. Sam’s selections were more “classic.” She likes to pretend that she is going to make something boring. It is part of her sewing scam job as you’ll see later.

I had awesome fabrics. I had a simple pattern. I had a great instructor. I was ready to quilt!

Week 2: Homework

This was the first class with real work. Susan lead us through cutting, ironing, and then more cutting and ironing. It looked like this whole thing was just a quilt-load of cutting big things into little things and then ironing them flat.

By the end of class, Susan performed some occult magic where a bunch of cut-up strips of fabric were sewn into a tic-tac-toe called a 9 patch. It was like watching an origami master pull a crane out of a square of paper. “It’s simple” she promised.

Sam produced her first 9 patch in 15 minutes. “Fantastic corners” Susan enthused as she held up Sam’s 9 patch to show the rest of the class. After about an hour of polite swearing I managed my own. The class was wrapping up and Susan drops, “Ok, so you are finished with your first 9 patch. For your homework, you need to have 36 finished by next week.”

Thirty six!?! At one hour per patch, I was seeing my evenings vanishing before my eyes. Sam was smiling away at her stack of 5 completed patches.

Week 3: Maim and Betrayal

The first thing that Russel, the owner and main pusher, explained in my very first class was that if you are going to use a rotary cutter, you should get a ruler with a guard. A guard cost a few dollars, he reasoned, and were huge bargain compared to the cost of stitches. While swearing my way somewhere between my 28th and “Eff This” 9 patch, I managed to grab my non-guarded ruler.

Completely over it, and completely over confident, I actually remember thinking some snide nanny comment as I sliced right through my fingernail and my thumb.  My first thought was to not stain the fabric with blood. I may be a fool, but I am not wasting this fabric!

In the coming weeks, the gash in my fingernail kept catching everything, so I kept a serious bandage one it. Bandage plus 30-40 hours of fabric pulling a week resulted in a huge blister on the pad of my thumb which eventually popped. (Don’t stain the fabric!).

It is in this week that you cut out twenty five 8.5 inch squares of fabric which are to become the central pattern you see in the picture. I had two fabrics that I thought I could work into twenty five unique squares. Everyone else had simple patterns and were aiming for their squares to all be the same or very similar. I spent a whole day struggling how to get twenty five unique cuts from a set of patterns that had no intention of conforming to my 8.5 inch squares. I cut some on the bias as I was running out of possible selections.

Then I spent a day arranging and re-arraigning the twenty five squares. I didn’t want two similar ones right next to each, or on top, diagonal, etc. The bias ones needed to be in the corners, with the angle perpendicular to the corner they were in. It was like trying to do a seating chart for a wedding. Aunt Gertie can’t sit by Moisha. Mary has terrible BO and needs to be put in a corner. Finally I got my dysfunctional fabric family laid out ready for assembly.

In class, Sam announced that she didn’t like her “classic” fabric. She went on some sort of underground rail road to the promised land of Asian Discount Fabric Warehouses with her mom where Mamma-Sam found some amazing owl (*AHEM*) fabric. This particular fabric was made for quilters, with designs conveniently arranged in the standard 8.5 inch squares. Sam was way ahead of me in work, and now also way on trend…my trend!

Week 4: Staying in the Lines

Quilting is essentially taking a few large pieces of fabric, and obsessively cutting them into smaller and smaller pieces like a little kid who just keeps tearing a sheet of paper in half over and over again. Then, unlike that kid, you put all the little pieces together to make a single even larger piece of fabric. Susan ensures us all that this is an easy and enjoyable process. Her nordic charms and patient instructions makes the 2.5 hours each Sunday seem easy and enjoyable. The other people in the class are taking each step in some zen-like stride.

At home it is a different story. Basically I am to take eleven squares and sew them together in a strip. I will in turn make 11 of these strips. Then, because I measured so carefully and sewed so evenly, when these strips are sewn together everything will line up into a clean grid. I was sewing with just one good thumb I remind myself as my grid looks more like a drunken hopscotch game. My borders were off like China meets Tibet.

Think of my book reading and tea, I pleaded with myself. I ripped each crooked seam out and restitched them one by one. By an act of will that would have made my pioneer women quilting fore-mothers proud, I made those squares fit! Mostly!

At class, Sam didn’t just show up with her strips done. She had the whole freakin’ quilt top done! When I said she is a competitive person, she is not really a person. She is some sort of Perfecto-Bot assembled in a military grade clean room facility in the Far East. Her handwriting can replace most laser jets, she can sew seams within a micron, she can thread a needle using just her mind!

She is the kind of friend I desperately need. I have a fear of needing emergency medical attention while on a plane or some other place where a doctor couldn’t get to me. Just look at my thumb, I am a walking HMO nightmare. She is the kind of girl who could get on the pilot’s radio and be talked through surgery. She could save my life and give me stitches that would probably shame most doctors.

I look at my puckered strips and then over our table at her Home and Garden’s photo shoot waiting to happen. The other people in the class come by to bask in the glory of her quilt. It does look amazing. Back off jennie-come-lately,  I found her first! As she smiles and explains her simple tips, I scan her neck, suspicious that there is some small, hidden button that when switched will either cause her titanium skull to open and reveal some sort of super-computer or launch all of Earth’s nuclear missiles, causing Judgment Day and the eradication of all humanity at the hands of a race of smiling, quilting, Terminators. Awesome!

Week 5: Feel the Burn

For the first time in weeks, I am sewing without bandage or blister. My thin-skinned thumb is a little sensitive, but I chalk that up top the fact that it is seeing sunshine for the first time in weeks. It feels good. We are in the home stretch and all that is left is for me is to put the border on. Sam, of course, finished her quilt already and just came to class for support, and to make super-cute pin cushions to the rest of her new friends.

I am stretching out the long strips of fabric that I a going to “miter” together and make the border for the quilt. This will involve a final act of Susan’s occult fabric origami, which I striving to understand. Thinking about this flippy, foldy, fabricy thing she does, I stretch out a strip of fabric and manage to place the pad of THE thumb right on the iron behind me. I would have been angry if I didn’t feel so stupid.

“Excuse me,” I yelped as I ran out the door. Stay calm I thought, as I watched what remaining skin I had swell and separate from the thumb. Next block over is a Trader Joe’s, closest probable place for ice. I run in, surviving the lunatics in the parking lot. “Do you have ice?” I urgently ask some calm looking employee.

I want to slap him for looking so happy with his job. I am clinching my finger and my teeth. He considers the question for a bit and finally manages, “I don’t think so.” Of course not! It’s Los Angeles. Where would Trader Joe’s be able to get locally sourced, organic ice from here? If only I was in the Anchorage store. I run into aisle three and throw my hand into the frozen food case. Instant joy! It felt so good, I could tell because people were looking at my face. “Who’s that pervert pawing at the sherbet?”

I groped around, looking for the frozen treat that will keep cold the best. I settle on some peas. On sale, bonus! I manage to get through the checkout without having to take my thumb off the package and walk back to class feeling all kinds of McGyver. Once back, the worried looks remind me that I am a mess. I am desperate for that emergency surgery from Sam, but she is engaged in making the last two pin cushions so everyone in the class can get one. No Sam love, and probably a good thing. I am sure that she would have amputated.

I wrap the package of peas onto my hand and start using my free elbow to feed the heavy quilt through my machine. I am going to finish this thing and figure out Susan’s final secret of the border. I am gritting my teeth, slimey pea juice is leaking down my arm, and I feel like Rambo with a sewing machine.

With just minutes to go, I fail. I ran out of time. I have all but the last 12 inches and then the magic origami. I feel defeated, but relieved, because I just want to put my hand in a bucket of ice and go to bed. Susan comes over to me, she had made a mini quilt with the super-secret border fold open for me to see. She gives it to me to take with me so I can finish my quilt at home and use it to see how it is done.

 Thanks Susan! Next weekend, with a good thumb, Sam came over and helped me finish my quilt.

Stupid sed Tricks: LDAP

Spent too many minutes doing a simple task today:

Take groups from LDAP and tell me who in in group 1 but not group 2.

 

Apache’s Directory Studio is essential if you do a lot of LDAP work. It makes it easy to navigate and peek around. With this I was able to dump two files, each listing the members of their GroupOfName records. Each line looked like:

mail=emailaddress,ou=Groups,dc=company,dc=com

All I really cared about were the email addresses. So let’s get those first:

cat userlist1.txt | sed 's/mail=\(.*\),ou=Group,dc=company,dc=com/\1/' > emails1.txt

I did that for twice, once for each file. Then I wanted to sort them:

sort email1.txt > sorted_email1.txt

Once again, twice. Once per file. I need to sort them for the comparison tool, as it expected ordered data. Finally, my in group 1 but not group 2 report:

comm -23 sorted_email1.txt sorted_email2.txt

The comm command reports three colums: only in file 1, only in file 2, and in both. The -23 switch suppresses columns 2 and 3.

 

So it took me a good 5 minutes (after 10 to remember sed syntax), simple unix tools saved me a script. And if need to be done enough, could easily be a script. Yay Unix text!

 

Oppose H.R. 3025: AKA Cellphone RoboRape

 

Dear Honorable Congressmen and Congresswomen,

Today I ask you to oppose H. R. 3035: To amend the Communications Act of 1934 to permit informational calls to mobile telephone numbers, and for other purposes.

This bill would strip citizens of the protections we have from abusive phone calls from companies. These onerous calls, which invade into a our private lives, family times, etc have long been blocked on land and cell lines.

This is a thinly veiled attempt to circumvent these long standing consumer protections. Today, many Americans depend on cell phones*. Instead of arguing that companies should have freer access to people via cellphone, I would argue the opposite. Cellphones are usually on the person at all times and thus much personal and critical.

Instead of just interrupting our dinners with Families, cellphones would allow solicitors and “robodialers” to invade our workplaces, our meetings, our commutes, our schools, public places, etc.

Many of us use our cellphones for critical contact; parents with children, doctors or professional; we live “on call.” Giving the freedom to robodialers to invade that channel of communication would greatly diminish the utility of cell phones for citizens and bring them few if any benefits in return. This only benefits corporations, which unlike citizens, do not have rights.

When your cell phone rings at 1:30 in the afternoon, as a parent you know it is an issue. When you phone rings at 9:00pm at night, as a professional you know that something important at work needs your attention. If this bill passes, you will now rush in a panic to wait through 15 seconds of silence for a recorded message inform you of your opportunity to get yet another Visa card.

I for one do not pay $79 a month for what my family calls “The Bat Line,” to subsidize yet another advertising venue.

* 91% (Arstechnica, March 2010) use cell phones, and a Reuters survey reported that 25% of Americans only use a cellphone.

Eurozone North and South

 

I was listening to an interview with the editor of Germany’s “Die Zeit” as he opined about the “Protestant North” and the “catholic Club Med” South of Europe. His contention was that the Northern countries in the Eurozone were traditionally more fiscally responsible and subsidizing the free-wheeling and spend more than you make Southern states.

The question at hand is, can they dissolve the Eurozone. Like the USA, it is a federation of independent governments that joined to create a greater economic/political power. Like the antebellum America, there is a debate about if you can freely join a union, can you freely leave it?

Lincoln realized that America would never be a serious nation and the federal government would be impotent to make tough or timely leadership if states could leave when they didn’t like what the union had decided. A permeable union would have no clout or authority, so to preserve the states and the union for those who valued it, he decided that we were compelled to keep it intact and enforce the authority of the central union over the member states.

The EU struggles because that question still lingers. Can Greece decide not to pay, can the North split off on it own. Like America’s civil war, it was more sparked by two different economies running at two different speeds roughly forced together than anything else.

Is the EU at a similar crisis point because of a similar economic situation: two different realities trying to rationalize as one. If you look at the employment and poverty rates in the American North vs. the South, you can see that there is a lot of work remaining after 100+ years. Do the relatively pampered Europeans have the stomach for what lies ahead?

Ideas evolve

A recent Smithsonian Magazine article, What Defines a Meme, gave a quick history of the concept of a Meme. The Information-Generation might like to think that they were invented along with the Internet as a way to procreate punchlines that cannot survive without context, but not so.

The idea of a Meme is older, dating back to when scientists tried to describe how information evolved and procreated like biological organisms. It is an idea that has always resonated with me. Life is nothing more that information that has found a way to procreate itself in the physical world. A meme would be information freed from a particular physical representation, but still able to procreate.

Where your genetic information needs base pairs to replicate, religion and nationalism are able to replicate at the informational level without a physical analog. I am attracted to the idea that human beings are carriers of memes the same way you could argue that we are carriers of DNA.

Without the error-checking of a physical construct, memes mutate and evolve more rapidly than genetic information. In the last century of scientific progressiveness, we often mistake evolution with progress. Evolution is actually a nasty affair of mutate and hope for the best. More times than not, life evolves to fill doomed niches, isolated corners, or the role of a disease.

In survival of the fittest, it isn’t the more correct or useful meme that survives, it is the one that is the best suited for replication between humans. Simple ideas that are easy to communicate and remember seems to be better than nuanced reason. Ideas that reinforce our prejudices are more easily injected into the host.

In this era of “I read it on the Internet,” and Fox News; I fear that our rapid communication is just allowing the most brutish of memes to spread the fastest. Just like evolution, rate of transmission and replication makes genetic data successful. There are no other criteria of worth.

In biological arms races, the fastest replicators choke out the slower organisms. They fast reproducers quickly gobble up all the resources and starve off the competition.  In keeping with a biological theme; this rapid proliferation and adaption of memes could be creating super-memes, or reason-resistant memes. In short, do our human heads have enough room for all the stupid memes? Or, like super flu-bugs, will the infection be too much?