Last year, Google added a feature so that you could send authenticated email through another SMTP server. This eliminated the need for the "on behalf of" Sender header.
It requires you to enter the authentication details of an account and it verifies that it can access the SMTP server at that time.
However, if that account changes (i.e. password change or anything which would cause authentication to fail), you will get no notification of email failures. You can click send and the message will go into sent items, but will never arrive at the destination. You will receive no bounce. You will receive no message from Google that the account isn't working, there is no visual indication that anything is wrong.
Your mail will simply and silently not be sent.
2010/12/14
T-SQL Tuesday - Business: Listen and Learn
T-SQL Tuesday Hosted by Steve Jones
The official topic this month is:
What issues have you had in interacting with the business to get your job done.
I have entitled my essay: Listen and Learn
I have found that most things in life obey an 80/20 rule. With regards to this rule, business is no exception. I find that dealing with the business, like dealing with anything else, requires you to listen carefully and understand not only the requests and the rationale for them but also the personal motivations of the people behind the requests.
There are many myths about "business" people - the first is that only "the business" is good at business. Well, only about 20% of business people are really good at business regardless of how you define it. And even out of those, there are other factors besides skill involved. A lot of business is not "business" - it's luck and human nature. And plenty of people who are "good" at business aren't "successful" at it, and plenty who are "successful" but aren't "good" at it.
Look at Steve Jobs. His "business" skills are product-oriented. The things he is good at are not financial problems, or production problems, or operations problems, or any number of other things you may associate with business. He's not a deal-maker CEO, either. This is a completely different motivation and focus than the next business person you may pick.
In the CEO or President role you will see different motivations than in the CFO or COO or CIO roles. Being aware of that is a key to being able to present solutions and strategic options.
IT is just part of the business and the users in IT who understand the business and understand the motivations will succeed beyond those who treat IT as a compartmentalized endeavor. If a CEO has weak technological skills, they will need to be augmented by someone who can provide technology options which can inform, enable or drive strategic initiatives. Even in an enterprise which is not technology driven, some technology (even if it may be considered outdated, like paper and pencil) will be used.
It's important to have IT be a strong partner in the business, through and through. For that to happen, IT people have to listen and learn the business completely.
Because some day you may be running the business.
What's up with Gawker and why it's a big deal?
On Gawker's side:
Amateur security fuckups - plain and simple - these are all basic security failures
You shouldn't be storing encrypted passwords in the first place - even if the data is encrypted, what are you using for key management? - is all the data encrypted with the same key? Why worry about that - DON'T STORE PASSWORDS.
When you store a hash, store a salted hash to avoid identifying people who have the same password and help to avoid the easy attacks
They didn't notify any of the affected users until days after twitter was already full of information about it
On users' side:
We all know you can't remember a million passwords - you have to use a password vault software and where possible use central authentication like facebook connect or OpenID.
You can't re-use passwords - look at the site - Gawker is a multi-million dollar enterprise and they are security fuckups - how many sites do you think may not be even encrypting passwords
Looks like it might be a good idea not to re-use email addresses either, since the problem cascades with using the email address to identify users fairly uniquely - this is where the whol system breaks down (see my paragraph later about infrastructure)
When you go to a site you rarely use like Gawker or an online forum where you need help with a question about bicycle repair - take the time to make a unique email address and password for the site instead of using a throwaway weak password you use for lots of unimportant sites. Because even if all those sites are meaningless to you and your important accounts are secured with strong passwords, you still can be majorly inconvenienced when your email address cascades through the systems and the owners decide to desiable your account because a site like Gawker is hacked. In addition, you'll have a good record of all these little accounts and be able to go back and check them. If you've taken to using the same email address and password for all these little sites, you'll have trouble finding their details, but with a password vault they will all be there for you.
Other sites:
It's nice that you have a list of affected users' emails - the effort to disable all their accounts and require resets is a good one. So far I've received notification from facebook, Digsby and LinkedIn. If you could quantity that cost and bill it back to Gawker, that would be great - because now even users with unique secure passwords on different sites have been inconvenienced by the fact that they didn't want to also have to remember a million email addresses so they re-used their email address for a ton of sites.
Too bad lots of other sites won't run the email list through their systems and notify their users
On the infrastructure side:
More than ever, it's clear that we need sites (especially those which don't have the resources to follow basic security principles) to move to OpenID, facebook connect or whatever, and that users need better tools to manage their digital identities over thousands of sites.
Even with a password vault, we are already managing too many user ids, emails and passwords, all with varying standards for strength.
Amateur security fuckups - plain and simple - these are all basic security failures
You shouldn't be storing encrypted passwords in the first place - even if the data is encrypted, what are you using for key management? - is all the data encrypted with the same key? Why worry about that - DON'T STORE PASSWORDS.
When you store a hash, store a salted hash to avoid identifying people who have the same password and help to avoid the easy attacks
They didn't notify any of the affected users until days after twitter was already full of information about it
On users' side:
We all know you can't remember a million passwords - you have to use a password vault software and where possible use central authentication like facebook connect or OpenID.
You can't re-use passwords - look at the site - Gawker is a multi-million dollar enterprise and they are security fuckups - how many sites do you think may not be even encrypting passwords
Looks like it might be a good idea not to re-use email addresses either, since the problem cascades with using the email address to identify users fairly uniquely - this is where the whol system breaks down (see my paragraph later about infrastructure)
When you go to a site you rarely use like Gawker or an online forum where you need help with a question about bicycle repair - take the time to make a unique email address and password for the site instead of using a throwaway weak password you use for lots of unimportant sites. Because even if all those sites are meaningless to you and your important accounts are secured with strong passwords, you still can be majorly inconvenienced when your email address cascades through the systems and the owners decide to desiable your account because a site like Gawker is hacked. In addition, you'll have a good record of all these little accounts and be able to go back and check them. If you've taken to using the same email address and password for all these little sites, you'll have trouble finding their details, but with a password vault they will all be there for you.
Other sites:
It's nice that you have a list of affected users' emails - the effort to disable all their accounts and require resets is a good one. So far I've received notification from facebook, Digsby and LinkedIn. If you could quantity that cost and bill it back to Gawker, that would be great - because now even users with unique secure passwords on different sites have been inconvenienced by the fact that they didn't want to also have to remember a million email addresses so they re-used their email address for a ton of sites.
Too bad lots of other sites won't run the email list through their systems and notify their users
On the infrastructure side:
More than ever, it's clear that we need sites (especially those which don't have the resources to follow basic security principles) to move to OpenID, facebook connect or whatever, and that users need better tools to manage their digital identities over thousands of sites.
Even with a password vault, we are already managing too many user ids, emails and passwords, all with varying standards for strength.
2010/11/02
T-SQL Tuesday: Why are DBA skills necessary? I pick the most important one...
T-SQL Tuesday Hosted by Paul Randal
What are DBA skills in the first place? There are so many skills which are generally useful in IT and particularly in the position that DBAs find themselves in.
I am not a DBA. I have done part of the DBA job before. I have been a developer and also an IT department director. I currently view myself as a systems architect - since I cover the gamut of knowledge necessary from infrastructure to building blocks to programming to database design to human factors and business (re-)engineering. Over the last 17 years, I have done a lot of OLTP database work and several years of OLAP work.
If you look at the ability to understand and organize complex systems, or the ability to understand a particular platform well, or to communicate well with business stakeholders, IT management and development staff there are a lot of skills which a good DBA is expected to have. I've decided to pick a single particular technical skill which I think it is very important for many people in IT to have, but extremely important for DBAs and network administration staff to have. It's needed in a situation where developers and business analysts will come for help, and a need which can never be outsourced or automated out of a job (don't we all look to put ourselves out of a job anyway?). When you're in the hot seat, that single skill is one that I think I value most in a DBA.
It's really a composite skill which combines a variety of traits and skills, but when I tell you the name, I'm sure you will all agree that it is a well-defined skill that you will recognize as such: troubleshooting.
When you are troubleshooting, you're in a particular mindset. Everything is up for debate. Is the hardware even working right? Is the network even connected? Is the cable bad? Did something change? It often starts as a controlled panic.
It takes a combination of knowing the platform itself thoroughly, and knowing not only how it should be behaving, and how it is actually behaving, and knowing how to find out how it is behaving. Being able to pull out network analysis, database and OS tools to monitor queries and the results. Being able to profile, being able to test ad hoc versions of queries and test data. Knowing the expected data profiles. Knowing what to throwaway completely to avoid wasting time and what to keep in the back of your mind as the next likely candidate to try. And never dismissing anything too soon, lest it come back - because the problem is always in the last place you look.
This all happens quickly in some environments. Even if time is not of the essence because the system is down, all the time used is some opportunity cost. There is nothing better as an IT director than knowing that you have a good troubleshooter working the problem, that he/she is going to come back to you with an answer (even if it's "I don't know yet, but we're going to find out"), that they are going to know the problem inside and out and own it and eventually come back with not only a solution, but with a plan to mitigate it in the near term and long term, perhaps including monitoring in some kind of proactive way.
It requires tenacity, a capacity for absorbing the system operations fully and synthesizing the interactions causing the problem. It generally requires experience and exposure in general systems work, as well as an understanding of the particular system being worked on. And in the end, it may require humility and forthrightness if you are troubleshooting your own work. Troubleshooting a developer's work lets the DBA know what's coming, lets them help to guide the developer away from dangerous areas and shows developers that DBAs can be a useful force, not just an impediment or obstacle. But no matter what, being good at troubleshooting is tremendously valuable.
Having good troubleshooters on your team is vital. It's a skill which is vital for DBAs to cultivate and improve and it's a reason you have DBAs in the first place, whether you call them that or not, the people with those DBA skills are necessary. That's why I want someone with DBA skills on the team and out of all those skills, the one I value most highly is troubleshooting.
2010/10/01
Back in the Machine Room Again - Logically and Physically Grounded
When I first started out in software as a full-time permanent professional programmer (as opposed to the years of practice as a hobbyist and intern) in 1993, we didn't have much of a server, and Lantastic wasn't exactly much of a network. But we got the job done and built some great software (in C for handhelds). We used SourceSafe (before Microsoft bought it) and we had version control and builds and testing and everything which is apparently still not used in some shops these days.
When 1996 rolled around and I entered what you might call the business computing environment, we actually had a "machine room". It had its own air conditioning. There were only a couple machines in the room - a huge HP UX machine and a Windows NT 3.51 machine running Tomcat for a BBS/FTP/web server. There was a rack for telecom equipment, but it wasn't bolted down and the floor was carpeted! (static was always fun) Over the years, I guided that little room through a new tiling (delft blue was my choice), an expansion (and moving of the door) and eventually to complete retirement as we moved our data center to a dedicated facility (and eventually to another site after that). I remembered Henry who was working behind a rack one day when a KVM switch fell on his head, leaving a lot of blood on the floor.
For the last three years at the bank, I never got to go into the data center. Neither data center we used was even in Louisiana. I didn't miss it that much because I hadn't been in a data center in a while at OCA/Orthosynetics. We had all become quite detached from the hardware over the years. I guess that's an unforeseen effect of the cloud. For other people, I imagine that computing power has been like that for a while. Those people never had to think about cooling or cabling or power. I wonder about the kids these days - if the cloud will mean their whole impression of computing and what computers really are will be different. Computing was never abstract to me. Sure, there are aspects which are theoretical, but it is always grounded in a physical implementation.
Sometimes people in database design talk about logical and physical designs. It's so funny, because the physical design is really just another logical design in terms of the vendor database. And then even that translates to more logical designs, and even storage now is not physical. And then eventually, after a lot of SAN controllers and stuff, you eventually get to a bit on the disk, which is a logical version of (finally) some magnetic orientation of (physical) atoms. It cracks me up when people talk about clustered indexes in SQL Server being the order on the disk - dude you are so far removed from knowing what order things are on the disk, you can forget about even talking about disks - you could call them green clovers or blue diamonds for all it matters!
I was thinking about Henry (R.I.P. - I miss you) today when I went into the machine room at my new client engagement yesterday. First thing I looked at was the A/C - they've got a new unit - very shiny. It has pressure gauges. My first question was about the building water supply - open or closed? I remembered back at Lakeway where we had to ensure that the filters were cleaned regularly of insect carapaces which came into the water in the reservoir at the roof. Because they have direct pressure gauges, they can see problems like high head pressure directly (we used to get them reported on the Liebert's display). The racks were not as full as what I was used to and they'll be shrinking with more virtualization. Their SAN was only part of a rack, and they don't have a switch which takes up its own rack. Although I'm not ultimately responsible for hardware and network any more, it was great to be able to talk about their network architecture and stay grounded.
When 1996 rolled around and I entered what you might call the business computing environment, we actually had a "machine room". It had its own air conditioning. There were only a couple machines in the room - a huge HP UX machine and a Windows NT 3.51 machine running Tomcat for a BBS/FTP/web server. There was a rack for telecom equipment, but it wasn't bolted down and the floor was carpeted! (static was always fun) Over the years, I guided that little room through a new tiling (delft blue was my choice), an expansion (and moving of the door) and eventually to complete retirement as we moved our data center to a dedicated facility (and eventually to another site after that). I remembered Henry who was working behind a rack one day when a KVM switch fell on his head, leaving a lot of blood on the floor.
For the last three years at the bank, I never got to go into the data center. Neither data center we used was even in Louisiana. I didn't miss it that much because I hadn't been in a data center in a while at OCA/Orthosynetics. We had all become quite detached from the hardware over the years. I guess that's an unforeseen effect of the cloud. For other people, I imagine that computing power has been like that for a while. Those people never had to think about cooling or cabling or power. I wonder about the kids these days - if the cloud will mean their whole impression of computing and what computers really are will be different. Computing was never abstract to me. Sure, there are aspects which are theoretical, but it is always grounded in a physical implementation.
Sometimes people in database design talk about logical and physical designs. It's so funny, because the physical design is really just another logical design in terms of the vendor database. And then even that translates to more logical designs, and even storage now is not physical. And then eventually, after a lot of SAN controllers and stuff, you eventually get to a bit on the disk, which is a logical version of (finally) some magnetic orientation of (physical) atoms. It cracks me up when people talk about clustered indexes in SQL Server being the order on the disk - dude you are so far removed from knowing what order things are on the disk, you can forget about even talking about disks - you could call them green clovers or blue diamonds for all it matters!
I was thinking about Henry (R.I.P. - I miss you) today when I went into the machine room at my new client engagement yesterday. First thing I looked at was the A/C - they've got a new unit - very shiny. It has pressure gauges. My first question was about the building water supply - open or closed? I remembered back at Lakeway where we had to ensure that the filters were cleaned regularly of insect carapaces which came into the water in the reservoir at the roof. Because they have direct pressure gauges, they can see problems like high head pressure directly (we used to get them reported on the Liebert's display). The racks were not as full as what I was used to and they'll be shrinking with more virtualization. Their SAN was only part of a rack, and they don't have a switch which takes up its own rack. Although I'm not ultimately responsible for hardware and network any more, it was great to be able to talk about their network architecture and stay grounded.
2010/08/18
Too Good to Be True?
I am available for new side projects now and full-time engagements on (or possibly before) October 1, 2010.
My resume is up to date and I've started to interview (with good feedback and results, but nothing offered in my range of skills and experience) and have started investigating some ventures with former colleagues too.
I don't have an ideal project, but I have an ideal niche at a company (or companies, if a few want part-time engagements) that needs someone:
- Who is smart and gets things done
- Who knows technology inside and out - with breadth and depth
- Honest, straightforward, accurate, fast, with proven track record
- Dedicated to building quality systems and team
- Who is a general systems thinker and understands systems
- Dedicated to the company, the product, the team and happy customers
- Who can talk to other technical people and be respected
- Who can talk to business people and understand their needs
It just sounds too good to be true doesn't it?
I think that's a problem, and I don't know how to solve it. Because for the last 16 years, I have really made a great impact on 3 companies in very different ways and different roles - all blending very ambitious technology with business needs and tight constraints on team, budget and time.
Here's a letter I tried with no response so far:
I noticed you recently advertised a position or came up on my radar as a local technology company who might be interested in some local talent - ME - about to come back on the market. I am expecting to be available on October 1 for new projects or a long term employment opportunity and if your company is a good fit, my skills and experience could be an extremely valuable addition to your team.
As you can tell from my attached resume, I have hardcore development experience and my stackoverflow rep (http://careers.stackoverflow.com/caderoux) puts me in the top 1% of that community. Being in the business, you know that great programmers aren't normally on the market, because they're always working hard making companies successful with great software, and you also know that there's an order of magnitude difference in productivity between the best and worst programmers, and it's all about aptitude - years of experience don't make a bad programmer any better. I live for making great software and great software teams and I know what it takes.
Right now, I'm looking to find the next long-term move that would be good for me. Once I've found it, I'm probably not going to be available for a while, but I might still be available for shorter engagements.
I appreciate your time - let me know if you want to explore whether we have a fit.
2010/07/06
Programming with Love Since 1982
Over the many years I've been in software, trends have come from out of nowhere and become mainstream and themselves been replaced with new trends.
There was the object-oriented jihad against the infidels of (even good) procedural programming. Then there was the writing solid code/code complete movement of defect reductionism and root cause analysis which eventually begat an evidence-based movement, while being eclipsed initially by an extreme programming discipline. Which then begat test-driven development and behavior-driven development, which coexisted with the great battle of waterfall vs. agile. All the while the Gang of Four were using patterns to find a Quality without a Name, and standard language for describing the pretty pictures was unified. And now, software craftsmanship and SOLID principles, all combined with an healthy injection of inversion of control containers, continuously integrating software to save the world.
I think at some point talking about programming got more interesting than building great software.
Ultimately, all those techniques, strategies and tactics, all the talk and methodologies, the architectures and patterns and practices: they mean nothing. The systems produced (both within software and around the software) either live or die. They may live for decades until quietly disappearing or they may be killed suddenly and dramatically before birth.
I was reflecting recently with coworkers on a system I helped build with a very small team (two programmers, two experts - our own Gang of Four). This software has lived now for over 12 years, providing a crucial operational system for a company I am no longer involved with. I would estimate its value in millions of dollars over that time. Like any child, it cost a lot of money to modify upgrade and maintain, many programmers and support staff, all putting a lot of love and frustration into the product. But like all successful software, it provided a tremendous value. It's gradually being decommissioned to be replaced with COTS software which has its own failings (and costs).
The love we put into that software was not love for the program - it wasn't terribly lovable code, nor was it sexy or exciting as code goes (although all programmers find joy in code). It wasn't love for the company - the business wasn't terribly compelling, nor would it change the world. I'll admit that I always thought that it was important to preserve the integrity of the system - that loyalty to the principles the software was founded would result in a successful product if those principles had value and the system embodied those principles.
But the system was not something one could interrogate to understand its purpose in life - we imbued it with life, there must have been something providing those principles. Sure, we were a business and we needed to provide value, but what brought those principles into focus is still somewhat of a mystery to me. Why did the software have a principle that the user should not be made to feel stupid? Why did the software have a no-modal dialogs policy? How much of this came from Alan Cooper? How much of this was important to the software's success and how much was incidental? Why did we have such strong feelings about how the users would interact with the software in as natural a way as possible? To this day, I don't know if this was just something about us, the team?
I strongly believe that there was a spark that the software we were building would help people achieve their goals. Knowing their goals, one only had to make their path to those goals easier and they would also eventually come to love the software, and own the future of that software. Because building software is about building things for people and when you make something for a person which adds something to their life, that is programming with love.
There was the object-oriented jihad against the infidels of (even good) procedural programming. Then there was the writing solid code/code complete movement of defect reductionism and root cause analysis which eventually begat an evidence-based movement, while being eclipsed initially by an extreme programming discipline. Which then begat test-driven development and behavior-driven development, which coexisted with the great battle of waterfall vs. agile. All the while the Gang of Four were using patterns to find a Quality without a Name, and standard language for describing the pretty pictures was unified. And now, software craftsmanship and SOLID principles, all combined with an healthy injection of inversion of control containers, continuously integrating software to save the world.
I think at some point talking about programming got more interesting than building great software.
Ultimately, all those techniques, strategies and tactics, all the talk and methodologies, the architectures and patterns and practices: they mean nothing. The systems produced (both within software and around the software) either live or die. They may live for decades until quietly disappearing or they may be killed suddenly and dramatically before birth.
I was reflecting recently with coworkers on a system I helped build with a very small team (two programmers, two experts - our own Gang of Four). This software has lived now for over 12 years, providing a crucial operational system for a company I am no longer involved with. I would estimate its value in millions of dollars over that time. Like any child, it cost a lot of money to modify upgrade and maintain, many programmers and support staff, all putting a lot of love and frustration into the product. But like all successful software, it provided a tremendous value. It's gradually being decommissioned to be replaced with COTS software which has its own failings (and costs).
The love we put into that software was not love for the program - it wasn't terribly lovable code, nor was it sexy or exciting as code goes (although all programmers find joy in code). It wasn't love for the company - the business wasn't terribly compelling, nor would it change the world. I'll admit that I always thought that it was important to preserve the integrity of the system - that loyalty to the principles the software was founded would result in a successful product if those principles had value and the system embodied those principles.
But the system was not something one could interrogate to understand its purpose in life - we imbued it with life, there must have been something providing those principles. Sure, we were a business and we needed to provide value, but what brought those principles into focus is still somewhat of a mystery to me. Why did the software have a principle that the user should not be made to feel stupid? Why did the software have a no-modal dialogs policy? How much of this came from Alan Cooper? How much of this was important to the software's success and how much was incidental? Why did we have such strong feelings about how the users would interact with the software in as natural a way as possible? To this day, I don't know if this was just something about us, the team?
I strongly believe that there was a spark that the software we were building would help people achieve their goals. Knowing their goals, one only had to make their path to those goals easier and they would also eventually come to love the software, and own the future of that software. Because building software is about building things for people and when you make something for a person which adds something to their life, that is programming with love.
2010/06/19
New Professional Blog
Having a number of blogs and domains (none of which are terribly organized or active), I have decided to start to consolidate professional material here on caderoux.com.
This blog will focus on my experiences as a professional software developer, data professional and IT manager and director. I'm taking a leaf from Brent Ozar's series on blogging and twittering.
I still plan to keep lastmagnolia.com for personal blogging, cade.roux.org for posting media like photos from my phone, and semicolon.net for code (but I might be redirecting that here eventually).
This blog will focus on my experiences as a professional software developer, data professional and IT manager and director. I'm taking a leaf from Brent Ozar's series on blogging and twittering.
I still plan to keep lastmagnolia.com for personal blogging, cade.roux.org for posting media like photos from my phone, and semicolon.net for code (but I might be redirecting that here eventually).
Subscribe to:
Posts (Atom)