Facebook Youtube Flickr LinkedIn

Norm Coleman Website Crash Exposes Database and Email Lists

28 Jan
2009
  Email   Facebook   Twitter

Post Updated: 3/29/2009

First off, I would like to thank to everyone.

I’ve decided to write a summary to give you my perspective two months after putting up this blog post. I have continued to add to it in hopes of making the big picture more clear for people who want to understand what happened.

I talk about why I put up the post, the political power struggle I didn’t want to be a part of, how the media took what I said and turned it into what they wanted and what I’m working on to bring about actual change so personal and financial data will be safer in the future.

Continue reading the summary


How ironic is this?  I was on Lifehacker today looking for the article about Eraser (program that securely wipes out files) and saw that January 28th is Data Privacy Day!  What are the chances of a security breach regarding data privacy being discovered on the very day that has been selected to raise awareness of data privacy?  Geeze!

Did interview with PJTV, conservative focused online media site (PajamasTV)

Article at ChannelWeb, Serious Security Flaw Discovered In Less Than 2 Minutes On U.S. Senator’s Web Site

Excerpt from resume of website developer who created Colemanforsenate.com website:

ColemanForSenate.com
* Developed a custom content management system from the ground up in PHP

New Video is up! Live: Coleman Question and Answer after The Rachel Maddow Show 3/14/2009 12:45am CST

Interview with Rachel Maddow Friday evening 3/13/2009 MSNBC
YouTube Preview Image

Interview with MPR Coleman warns donors after data breach (audio of me from the radio)

Blog Post MN Independent Coleman donors express ‘extreme anger,’ fear, worry after breach

YouTube video: How I Found Norm Coleman’s Website Database in 2 Minutes

Best quote to me on the phone: “I just hung on the secret service to talk to you” — unnamed reporter

Lifestream video : I explain what went wrong and answer questions about the Norm Coleman’s website

Interview with MN Independent Coleman’s site wasn’t ‘hacked,’ says IT pro who discovered donor breach

Blog Post at MN Independent Breaking: Coleman’s unsecured donor database revealed on Wikileaks

Blog Post Here Who is Searching Google for Norm Coleman’s Database?

So, it sounds like Wikileaks.org is putting Norm Coleman’s business out on the Internet.


What’s worse than losing a Minnesota Senate race?

Losing your website’s entire database, that’s what.  As if claiming your website was brought down by too much traffic wasn’t bad enough, Norm Coleman’s website received a second round of criticism when I found a database file sitting in a directory that anyone could download…

I first picked up this story from @Chuckumentary on Twitter about Norm Coleman’s office saying their website had been “inundated by tens of thousands of hits today – temporarily crashing the website.” Of course that got me curious as an IT consultant and I went to check it out.  Aaron Landry broke this story because previous website traffic reports and the location of the domain name didn’t match up.  Paul Schmelzer at the Minnesota Independent picked up the story which is where I first saw it.

Norm Coleman’s website crash revealing a database full of supporters is now known as Crashgate.

Curious, I wanted to see where the domain was currently pointing.  I used OpenDNS.com’s cache check to identify the current ip address of 208.42.168.251 and then loaded that address into my web browser.

Screenshot of opendns.com information for colemanforsenate.com

I had to see what all the fuss was about.  Was there really an attempt to bring down the website due to political unrest with these ballots in my state?  Were the allegations of a poorly coded website true?

What I got instead was a plain text listing of directories…

The Database of Norm Coleman

Wowza.  As I was tooling around in the directories, I saw a database file.  I thought, “That’s not right.”  I began taking screenshots and uploading them to Flickr.  I didn’t know what the database contained but hoped there wasn’t financial information in that database.  I figured it was a list of email addresses for Norm Coleman supporters and staff but I did not download it find out.  Did you download the database?

YouTube Preview Image

There is a term known as “Google Hacking” where you can actually search for files that people have on sites and ftp areas that have names like “passwords.txt”, “backup.tar.gz”.  Eeek!  Backups should be stored above the “root” folder that is shared out to the internet.  This is showing up because the server located at http://208.42.168.251 was not told to restrict directories from the web.

All photos are licensed under Creative Commons.
Norm Coleman database photos on Flickr

I wonder how much user information is in this database at colemanforsenate.com?

I began posting links to the photos on the blogs of the Minnesota Independent and Minpublius to bring awareness to what I had found.  Would I have done the same if this were a democrat?  Probably.  For me, it’s about computer security and data privacy, not about political affliation.

You can become Norm Coleman’s Website Admin

I will give them the benefit of the doubt and assume I was only able to get here because the website is not functioning.  Below you can see that I could enter an email address, name and password and if this site was working, it would create an administrator in the database.  I found similar files to edit and delete records as well.  Being able to write to the database like this from a form should require an authenticated and active session but I can’t see the code so I don’t know.

wow, is it this easy to create an admin account at colemanforsenate.com?

Indexing of directories is turned on

This is a security risk.  I would hope they have .htaccess files in place to restrict access to the admin directory and that index listings are turned off for the current site.

directory of colemanforsenate.com at ip address 208.42.168.251

Website errors show you configuration file locations

You see errors like this a lot on Joomla websites when there is a problem connecting to the database, there is a permissions issue on a file or when files are missing.

Incorrectly configured Linux server to blame? colemanforsenate.com

Missing log files

This directory is empty.  It doesn’t mean there are no log files (deleted?)

why is this directory for log files empty on the colemanforsenate.com website?

Site is down again

So, the site is being reported by OpenDNS.com as down again and I am getting the same info at DNSStuff.com too.

colemanforsenate.com is back down again accordin to OpenDNS.com

The moral of the story is that you should hire computer and website professionals who understand technology.  You should plan and develop a strategy for downtime and problems.  Don’t put all your eggs into one basket with one website programmer.  If he or she is hit by a truck (or something goes wrong on the website and they have no recourse to help you.

Resources to protect your data

Minnesota Law on Data Security Breach Notification, Statute 325E.61 – This describes what needs to be lost for a company to notify you and how they must go about doing it. Unfortunately, it seems a company can lose your full name, address, income, number of children and previous purchases BUT not be required to tell you. (Disclaimer: I am not a lawyer)

Data Security Breaches in the US 2005, 2006, 2007, 2008, 2009 – Check to see if a school you attended, a doctor you saw, an employer, your local Veterans office, your bank, your utility company, your library or even a hotel you stayed at is listed here.

Resources for website security

The Importance of Web Application Scanning – Acunetix makes an application that can scan websites for vulnerabilities.  There is a free version that will check for XSS (Think back to when Barack Obama’s website redirected to Hillary Clinton’s).

3 Common Website Security Problems – This article from Georgetown University  summarizes how issues on Norm Coleman’s site could have been addressed before “Crashgate”, especially this one on unsecured files and databases:

Unsecured files and databases

When setting up your web site or application, make sure that any files that contain data that is not intended to be public (such as information about people) are not located in public web folders. Do not place such files in folders with the belief that because you are not linking to them, a user cannot find them.

  • Files (such as Access databases) that are datasources for your application must be located in a non-web-accessible folder (the web_datasources folder in your hosting account).
  • Other files that contain data used by the application should also be located in a non-web-accessible folder.
  • Other files that contain non-public information should be placed in a folder that is access restricted using a .htaccess file or other web server access restriction.

Update 12:12am 1/29/2009

Folks, the directory listing for colemanforsenator.com has been replaced with a login box.  But…we know what’s behind the curtain now.

Login box replaces 205mb database on colemanforsenate.com

Update 5:40pm 1/29/2009

Stay tuned for video posting from the 1/29/2009 lifestream:

“Norm Coleman’s Database”

  • why the database was available
  • what it contained
  • how website developers and companies can work to prevent this from happening
  • and take questions from viewers

Update 11:11pm 1/29/2009

Number of hits to the post 54

Photo stats for the post
I wonder how much user information is in this database at colemanforsenate.com? 1,458 views
You can become Norm Coleman’s Website Administrator at colemanforsenate.com 290 views

Current rumors
The database contains social security numbers
The database contains credit card information (POST data)

Update 6:54pm 1/30/2009

Number of hits to the post 610
In-Progress Video of “Norm Coleman’s Database: What Happened and Why”

Post picked up on:
Politics in Minnesota – Epic recount website fail: One Dot One Dot One Dot One

Thanks to Ben for picking out the incorrect use of “then” when I should have used “than” in the header “What’s worse than losing a Minnesota Sentate race?”

FYI: If you enter a fake looking email address with your comment, I will probably not approve it. If you want to share something with me offline, use the contact page. Thanks!

Question from Dennis
What does “Awaiting Moderation Mean? Where’s my comment?

Answer
I did not publish your comment because there was NOTHING technical in it. I have published comments that:

    * indicate how they feel about the info being released
    * indicate how they feel about what I did as an IT person doing this
    * ask questions related to the technology aspect of the Norm Coleman database
    * share personal stories on how this affected them
    * thank me for my efforts
    * support me for taking initiative
    * judge, criticize and blame me for making the wrong choice

If you just want to harp on Democrats vs Republicans and Norm Coleman vs Al Franken, you should go to a political blog and do that.

Blog Widget by LinkWithin
  • edevlet
    You may have accidentally coined a neologism.
  • Ah for "CrashGate"?
  • Amanda
    Okay so I just found your blog today. That's cool that you're the one who found all this out! I live in Minnesota as well so I heard about this all over the news, but I never knew who was the person who discovered everything. Very very cool. :)
  • decora
    i have learned to always think twice before i post crazy stuff i find on the internet. the ethical questions are not so simple as they seem at first glance.

    if what you find is truly important to society, you can always keep copies, and then upload what you found, later.

    journalists have big databases of notes and documents that they dont show to just anyone... they have all sorts of guidelines and judgement calls they have to make about what they can reveal, vs what they cant, and the effects it might have on the lives of various people, including their sources.

    the hacker culture is a bit different from the journalism culture, thats imho maybe part of the problem with the internet and events like this. it wasnt the first event, (ayone remember when AOL published a bunch of 'anonymized' search queries?), and im sure it wont be the last event like this to happen.

    hopefully the computer professors will start taking a little advice from the journalism professors and start teaching this stuff in their courses.
  • adria.richards
    @Jachra,
    I really appreciate you taking the time to watch the video and post back to the blog with your updated comments. Everyone has a right to their opinion about my actions.
  • adria.richards
    @BeckiTrue,
    No I didn't know the server was at VISI.com. That's interesting. I suggested that the Norm Coleman office contact VISI.com to get a senior level person to look at the server when the problem happened.

    I wonder why they didn't ask VISI.com to review the log files. VISI.com support folks are very experienced. Maybe they were collocating a server there.

    I wonder if this supports the theory they were moving the website to a new location and I found the old server.

    There are so many unknowns to this issue..
  • @Adria Richards,
    Did you know the site was hosted by Visi? I don't think it is now, but it looks like it was when you took the screen shots. They have 24x7 phone support BTW.

    : dig -x 208.42.168.251

    ; <> DiG 9.4.2-P2 <> -x 208.42.168.251
    ;; global options: printcmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48770
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 2

    ;; QUESTION SECTION:
    ;251.168.42.208.in-addr.arpa. IN PTR

    ;; ANSWER SECTION:
    251.168.42.208.in-addr.arpa. 900 IN PTR v-208-42-168-251.mn.visi.com.

    ;; AUTHORITY SECTION:
    168.42.208.in-addr.arpa. 900 IN NS ns.visi.com.
    168.42.208.in-addr.arpa. 900 IN NS ns2.visi.com.

    ;; ADDITIONAL SECTION:
    ns.visi.com. 807 IN A 209.98.98.1
    ns2.visi.com. 807 IN A 66.254.98.138

    ;; Query time: 76 msec
    ;; SERVER: 68.11.16.25#53(68.11.16.25)
    ;; WHEN: Mon Mar 16 21:33:32 2009
    ;; MSG SIZE rcvd: 154
  • Jachra
    Adria,

    I watched the video, but that does not change my opinion. You still should tried to contact someone at senator Coleman's staff and his ISP.
    Your failure to do so is very unethical.
  • adria.richards
  • Jachra
    Adria,

    Although this a great find, I do agree with others that you should have acted more ethically. This was definitely not a responsible disclosure. Any exploit like this should be reported first.

    Yes, they should have hired a proper administrator in the first place. However, I am not surprised that they had to call the Secret Service because of some regulation.
  • adria.richards
    @Minnesota Central,
    Agreed. The real issue at hand is the development of insecure websites.

    If you bought a car that you could not lock, it would get stolen often.

    If you adopted a puppy that was not vaccinated properly, it would need to see the vet.

    If a website collects information from people, especially financial information, it is a must to plan out the data workflow.

    Instead of storing the credit card information, they could have just collected the name, email, address and sent the financial portion onto a payment gateway processor like Authorize.net.

    The biggest problem here was the management of the server and website. The Coleman office could have hired an experienced Linux and website administrator to lead the website rollout, audit the site and read through the error logs. Instead, they brought in the "Secret Service" who found "nothing".
  • Just ran across your posting and as a non-technical person I must thank you for making your presentation so understandable.
    AND for sounding the alarm.

    I don’t know if you can tell from the information available, but I have a concern of a potentially bigger problem.

    There are some important questions that need to be asked :
    What company did Coleman hire to collect his donations ?
    Did that company perform similar work for others ?
    If so, does(did) that company maintain “illegal” information on their databases ?

    If the company maintained this information for the Coleman campaign, was the same information maintained by other campaigns ?
    The Coleman incident may have exposed a problem that every political campaign needs to address. Proactively, every campaign that collected monies through credit cards needs to perform an internal investigation and issue a press release if illegal information was maintained. This would include not only Minnesota campaigns, but also others that collected monies within the state such as Romney, Guiliano, et al … as well as Democrats.

    Admittedly, those other campaign sites may not be exposed yet, but that doesn’t mean that they are not maintaining information that they should not. There is no reason for waiting for the FEC, FBI, Secret Service or MN Attorney General to investigate … campaigns need to be forthright and transparent.
  • o4tuna
    @Dean W., For God sakes! Put down the Anne Colture books, turn off Fox, go buy a nice cd of polka music to play in your car. Get a grip before it is too late! Let rational thinking and vigorous curiosity return to that place it has long since vacated.
  • o4tuna
    @Bob, See my reply to TJSwift. You are another person who doesn't have the knowledge to make this judgment. She did nothing wrong. You sir, are completely ignorant of what she did, even though she has laid it out for you to see. Instead of asking questions, you choose to pass judgment and remain ignorant. This is your mistake. One in which I'm sure you often repeat.
  • o4tuna
    @TJSwift,TJSwift, you really don't have any idea what your talking about. Entering an IP address into a browser is not illegal or uncommon. And finding a Direcroy structure when you get there, and looking through it is legal and common too. it is assumed that if you find a directory structure, you can look through it. This is how it was back in the early days of the internet, and the practice is still used.

    What I find questionable is that there was a tarball (a zip file to you) of the database in a publicly accessible directory. This is either a huge mistake made by a complete beginner, or a plant. There are plenty of way to have kept this information, this file, from being found, by the method in which Adria found it. (It shouldn't have been there in the first place). So much so that I believe it was done on purpose. Nobody, nobody, that is in the business of building web sites for a living would put credit card information out on a open directory. This smells bad.
  • JohnC
    @adria.richards,

    Adria – you couldn’t fix the problem, that is correct. It wasn’t your job to do that, and I get that. I understand the technical specifics of the problem, as well as the concept of data breaches. I have been doing information security for almost 20 years. I can’t agree with you that you kept people safe here – you heightened the problem. But this too, is not about me. But it’s fair to ask, what would I have done in this situation? If I was looking to promote awareness as you mentioned you were doing, I would have taken the pictures – blacked out any personal information and then I would have tried – endlessly – to have contacted the ISP, Norm Coleman’s office, the domain administrator, etc. and let them know of the issue. I could have promoted the awareness on my blog later, after they fixed it. But instead you attempted not to fix the problem – which would have been to contact them, but rather you stemmed the continued bleeding of data. Obviously someone was around as you pointed that they put a password on the page. And don’t get me wrong – I believe they were TERRIBLY negligent. How did you help?
    None of the people who you mentioned as the victims of past breaches deserved to have their information exposed. Those organizations that were negligent have suffered as they should, and Coleman would have as well – but stopping the issue was more important than going to press about it while it was still hot.

    I do admire that you are willing to put up posts that disagree strongly with you, as I do. I believe you have a great future ahead of you, but may I suggest you put this behind you as quickly as possible because it does not bode well at all for your ethics. There are better ways to make the same point you are trying to make…
  • adria.richards
    @JohnC,
    I couldn't "fix" the problem. The real issue here was negligence.

    -The location of the backups were being done "one level up"

    -Someone was hired to create a Content Management System (CMS) that intentionally stored credit card numbers and the 3 digit security code

    -Someone was hired to setup and maintain financial transactions for the website who did not do it correctly

    -Someone was in charge of the website the day it had problems and made decisions which left it up with exposed folders...they also left the database file there

    This is about security and keeping people safe. Not the specific people who were in the database but to raise awareness that this is a global problem.

    Please take a look at the thousands of security breaches at this website for schools, banks, hospitals, law firms, retail stores, government agencies, police departments, hotels, veterans homes

    80,000 current and retired New York City police officers should have never had their Social Security Numbers and direct deposit information stolen March 4th, 2009 by a civilian employee. The link above says they caught him because he disabled the security cameras when he stole the backup tapes.
  • JohnC
    @adria.richards,

    Adria - I agree with Becki, this was definitely not ethical. It's irrelevant that it was a political site (and I don't care who wins). It's irrelevant that they may or may not have had professionals managing the site. It is relevant that they exposed data. Someone with more intelligence about such an issue - and how to fix it - such as yourself, had two choices. One was to find a way to address the problem, to contact someone - ANYONE. From 7pm to 730, you discovered more and collected your evidence. Sadly, you took the 2nd choice. You didn't address the problem, you publicized it - even insisting on your website that it was okay to publish the pictures, "as long as you credit me". Wouldn't an IT professional such as yourself better serve the public and help the weak by fixing the problem? Your own website says "I like to help people". Who did you help? I might have missed it, but I didn't get the impression that the Coleman people ignored your pleadings, your evidence, your *desire* to help them. Instead, you went for the 15 minutes of fame in this 2.0 world, even giving props to Twitter. For what, getting you on TV? You used your experience and greater IT knowledge for personal gain. But this too will fade. The legacy you could have left for this event could have been an ethical one. It's too bad it won't be.
  • Al
    @Al, Darn, that didn't work. Stupid ASCII art :(
  • Al
    @Adria Richards, Okay, I can accept that...Then, perhaps a flower for your efforts?

    8""=""8' "88a88'
    .. .;88m a8 ,8"" "8
    "8"' "88" A" 8;
    "8, "8 8 "8,
    "8 8, 8, "8
    8, "8, "8, ___8,
    "8, "8, "8mm""""""8m.
    "8,am888i"' ,mm"
    ,8" _8" .m888"
    ,88P"""""I888888
    "' "I888
    "I8
    "I8_
    ,mmeem.m""i, I8"" ,mmeem,'.
    m"" . "8.8 I8 ,8" . "88
    i8 . ' ,mi""8I8 ,8 . ' ,8"
    88.' ,mm"" "8I88"m,,mm'"
    "8_m"" "I8 "
  • Nice work! I was surprised when you were on the Rachel Maddow show and said it was just directory browsing being enabled on the server.

    This was a theft of information but there was NOT any hacking involved. This was simple negligence on their part. Assuming they had not made the obvious mistake of enabling directory browsing. Here are some other mistakes they made:

    1. Storing database files in the web root is a security risk for this very reason. You should always store data files below webroot so it would not be accessible in the event of a compromise.

    2. You should NEVER have a administrative backend that is accessible WITHOUT a password.

    3. You should NEVER store credit card numbers in plain text, it should be encrypted. You are also not ALLOWED to store credit card numbers unless your server is PCI Compliant, and this server is OBVIOUSLY not (being PCI Compliant costs a lot of money, and as we can tell the Senator's site is a low-budget).

    4. Storing CCV number at all IS ILLEGAL!!

    5. Their site is (was?) vulnerable to SQL Injection attacks, which would have been able to EASILY retreive credit card data from their database even IF they did not have directory browsing enabled.

    The fact that he is trying to push this off on to hackers is ridiculous. This is negligence.

    For the record, I work in the web-hosting industry and deal with compromised sites/servers on a daily basis.

    ~Blake
  • @adria.richards,
    I agree with Pixelpusher. You found an unlocked door, walked in, looked around and took pictures. Rather than notify the owner, you chose to put a big sign in the front yard announcing that the door is unlocked and posted pictures of the contents for everyone to see. For a technology professional, this is an ethical question, not a political one.

    You said it yourself, you thought it was "news" and that is lens through which you filtered your decisions. Your desire to be part of a news story outweighed your duty to act responsibly, and you helped to expose sensitive personal data that might not have been otherwise. It doesn't matter that you were not the first on the scene.

    Sure, you didn't unlock the door or store information that isn't supposed to be stored unencrypted, but you did tell as many people as you could about the vulnerability, and did so before the door was locked. You could have publicized the negligent actions of the site administrator after the vulnerability was dealt with. You would have made your point without unnecessarily exposing people to identity theft or credit card fraud.

    It's an easy mistake to make given the current emphasis on instant communication, Internet fame and the view that data nearly valueless. If nothing else, this incident serves as another case study for Information Assurance and Business Ethics students.
  • Adria Richards
    @Al,
    Oh dear! Al, I don't think I can commit on this just yet. I'll take it to mean you really, really, really appreciate my efforts!
  • Al
    Adria,

    Please marry me! You are my hero.

    Love,
    Al
  • Jon
    @Adria Richards,

    Good work Adria.

    For the rest of you: I too am an IT consultant and *applaud* the way this was handled. *Somebody* had to shed light on this, and because Norm Coleman chose his career over the financial security of his donors, someone else stepped up to the mic.

    The fact is, they knew they had a problem that they were obligated to fix. They knew they'd exposed donor information and had an obligation to inform the people affected. They didn't. Until someone forced their hand.

    Again, Well done Adria!
  • adria.richards
    @E Nelson,
    Nice reporting!

    I think your article brings up a very valid point that the information was sitting out there for anyone to access due to the choices the website adminstrator at Norm Coleman's office made.
    Norm Coleman and Identity Theft Gate: Is Your Online Donation to Norm Coleman Safe?
  • E Nelson
    @Dennis, Dennis you apparently have difficulty with handling facts. Let me help you. FACT the security breach happened in January and has been know by the Coleman campaign for a long time. FACT as of two days ago they finally started letting donors know their information had been compromised ONLY after Wikileaks.org had already started emailing some of the donors alerting them to this. Do you think it is ethical of the Coleman campaign to NOT report this breach immediately as is required by the law??
  • adria.richards
    @E Nelson,
    Wow, your comment has blown me away.

    So much so that I turned it into a YouTube video!

    Check it out!

    YouTube Video
    Hey Bob, I've Got Something For You...Re: Norm Coleman Database
  • Dennis
    @adria.richards,
    Coleman's IT group apparently is not too savvy and made a mistake, which they could not find right away. Maybe they are volunteers and not professionals. Who knows?

    Regardless, what possible motive would Coleman or his supporters have for releasing this data? Absolutely none. This information is totally devastating for his fund-raising.

    On the other hand, what possible "ethical" motive would someone have to pick apart this website to find this information? Absolutely none. No matter how the information was used, or released, or reported is certainly not in Coleman's interest or his donors.

    Did anyone make an "honest" attempt to contact Coleman, before releasing this information to the world? From the sound of the correspondence, I seriously doubt it. Would a greater attempt be made if Franken made the mistake? I think I know the answer.

    The whole thing should have been handled privately. Exposing the error to the public was not ethical, and the self-righteous attitude about exposing the information is appalling.

    @Dave,
    I’m glad you are sending Franken $100 to help him pay for the $thousands$ he owes in taxes in more than a dozen states. Norm’s volunteer IT folks may not be a savy as they should be, but Franken is personally irresponsible and dishonest.
  • E Nelson
    Okay, I couldn't take it anymore after reading all the ignorant comments attacking Adria. I am a fellow IT consultant and I deal with security issues every day. I see hackers scanning my clients sites EVERY day looking for potential openings and exploits. These hackers are using untraceable zombie networks from all over the world. Chinese hackers, Romanian hackers and yes many pre-teen US hackers.

    Just because Bob and the rest of the computer illiterate posters here have no clue about technology does not mean that anyone else should remain as clueless as they are. The fact of the manner is an IT consultant responsible for my client's web technology and any sensitive information associated with their websites I ABSOLUTELY want an Adria to point this out as quickly as possible so that I can act on it rather than have multiple GIGANTIC security holes remain exposed for weeks with hundreds of untraceable IP connections downloading the information. All of these security breaches can be easily found AUTOMATICALLY with internet scanners very similar to what Google uses to index the entire internet. I hate to break this to you Bob but I can almost guarantee you that their are Chinese and Eastern European hackers that have had this information well before Adria found it. And if you think they are going to call up Norm and let him know I have some oceanfront property in Iowa to sell you.


    The fact of the matter is that Norm Coleman and the people working for him are either completely incompetent or blatantly negligent. Adria was not the first person to alert the Norm Coleman campaign to the potential problems and yet they continued to ignore their duties to A) FIX THE PROBLEM B) Alert the donors of their mistake and C) TAKE THE DAMN SITE DOWN. It takes 2 minutes to do this until you can figure out what the problem is. Instead the Coleman campaign claimed their site was hacked for political purposes, claimed they contacted the Secret Service to investigate and who then unbelievably and incorrectly said that no sensitive information had leaked out.

    So the question should be Bob, as a donor would you not want Norm or someone else alert you to the fact that your credit card information has most assuredly fallen into the hands of international hackers?
  • Dave
    @Bob, you're on to a great analogy there. This particular shopkeeper hired incompetent help to decorate the windows and they used all the customer files to do it. Nobody notices until one day the shopkeeper forgets his shop address, freaks out and alerts the media that his competition has obviously stolen his shop! A good Samaritan looks where the shop was the day before, finds it still there, notices the customer files posted in the windows, and updates the media on the real story. The shopkeeper then gets his friends to attack the good Samaritan.

    Yeah, I'm blaming the shopkeeper for all that. He's the only one throwing bricks in the entire town.
  • adria.richards
    @Karen,
    Thank you for your nice words!

    Cool! I did not see this play by play of how things developed. this blog post helped spell it out. I also made a video on YouTube "How I Found Norm Coleman's Website Database in 2 Minutes"

    You can hear me on MPR's broadcast Coleman warns donors after data breach where I say:


    "It's like putting your filing cabinet outside of your house"


    MPR "All Things Considered"
    Aired Wednesday 3/11/2009 at 5:20pm.
    Talked with Mark Zdechlik at approx 1:30pm. He recorded our conversation for the show.

    Thanks for the support. I really mean it. People, technology clients and friends who know me are supporting me on this. Although it was a racy move, I was frustrated yet again to see information that people expected to be secure, publicly available online.

    The goal was to create exposure of the issue and raise awareness; Security and Data privacy...not Democrats vs Republicans.
  • Karen
    Adria, well done. Have you read this: http://www.dailykos.com/story/2009/3/12/101429/... ??

    The writer put it in very simple terms: norm placed his donor credit card info and CVS numbers in front of his yard for anyone to look at. I've never heard of anything so dumb.

    Not to mention illegal. CVS numbers??? Not notifying donors?? Not using an independent campaign contribution link. Dumb.

    Good for you and I think you're being very gracious about the flak you're taking.
  • adria.richards
    @Pixelpusher220,
    I agree that notifying companies of exploits to their software and code is important and doing it in a way that does not comprimise data or users before a patch is developed helps everyone have a happy computer.

    I decided this was "breaking news" and that I was going to document my findings.

    I agree with @dan tynan that there is no proof I was the first one to find the server's files.

    I saw the tweet on Twitter at approx 5pm
    I found the files and database approx 7pm
    I posted photos to Flickr approx 7:30pm
    By 12 midnight, the site had been updated with a password screen

    You can see all the Norm Coleman database photos on Flickr
  • if anyone's angry, they should be angry at coleman's tech team (and his press/response people) not adria. they screwed up six different ways, at least.

    why were they keeping these credit card numbers? unless they were a) doing recurring charges, a la amazon one click, or b) credit card cramming, they had no business holding on on to them. let alone putting them on a server attached to the net, marrying them to names and addresses, and leaving them unprotected on a web site. stupid stupid stupid.

    and of course, if the breach was serious enough to bring in federal investigators, it was serious enough to notify the 4,721 people whose credit card info was exposed.

    the commenters here asume that adria was the first person to find these databases. there's no basis to believe that. we don't know how long the data was exposed or how many people saw it. if not for adria, the data would have been exposed for far longer (it might still be exposed), and those people would be at greater risk.

    for more on this, see my blog -- http://www.dantynan.com/2009/03/12/beyond-the-n...

    cheers,

    dt
  • Pixelpusher220
    "Do you really think if I’d called the Norm Coleman office at 7pm that night, anyone would have answered?"

    Well, maybe they would have and maybe they wouldn't have. But since you didn't do that, it's a bit specious to claim they *wouldn't* have.

    If you had taken your information to a credible source, such as a major newspaper...I bet Coleman would have answered *their* calls. Putting this information front and center in the public eye prior to giving Coleman's team a heads up wasn't the best display of ethics.

    If you found a zero-day flaw in IE or Firefox, would you call MS or publish your info to the web for it to be exploited?

    Go read how Dan Kaminsky handled the DNS vulnerability. He didn't show the world because he knew it would cripple the internet. He worked with people to solve the problem prior to publishing the full knowledge.

    In this case you didn't care that posting the info could cripple Coleman or his donors.

    disclaimer: I'm an IT person as well, and fully support Franken. I'm happy to see Coleman's lack of security broadcast. This just wasn't the proper manner to do it.
  • Wikileaks.org releases the ?entire? database. Someone must have not made them happy. Found at http://theuptake.org/
  • JustADude
    Did she actually publish the contents of the database? NO. Did she use this as an example of how to protect your website? YES. Did the Coleman people take the wrong steps in hopes of pointing to Democrats to make himself look better and help his failing case? Yes. Coleman is the culprit. He runs the shootin match and so he has to take the fall. Shutting down the site immediately, good idea. Leaving it open and calling the Feds, stupid and a bad idea. I hope the majority of the people that live in MN are not as dumb as some of the people that have posted comments on this. Sheesh.... Give me, and us all, a friggin break.
  • TJSwift
    @Angela, Yes, but it appears that out of that countless crowd, it was Adria that lacked the moral and ethical fortitude not to.
  • TJSwift
    @adria.richards, I will concede one thing regarding the capacities of five year olds.

    Most five year old children have a more mature understanding of right and wrong than I see displayed here, and elsewhere regarding this issue.
  • adria.richards
    Question from Computerworld reporter:
    Do you think the same thing would have happened had you not publicized details of your discovery (on the exposed files) on Flickr and elsewhere?

    Some are saying that if you hadn't publicized it, maybe none of this would have happened and that database wouldn't have been accessed and the personal details posted on Wikileaks. What do you think? Valid point? Or is it more a case of, if it hadn't been you, then someone else would have surely found it?

    My Answer
    Someone may or may not have found this. I did think it was better it to be me than someone else who would just "steal" the info and no one would have been the wiser.

    Maybe if there was a Federal or State escalation process, "IF YOU FIND PERSONAL DATA ONLINE, REPORT IT TO 1-800-911-DATA" or something, I would have felt I had a way to address this to the appropriate response team who could fix it.
  • adria.richards
    @schtum,
    I love that example!
    "...under a tarp"
  • schtum
    @Bob, What if your bank wasn't keeping your money in a vault, or even inside the bank, but under a tarp in the parking lot? If someone like Adria came along and said "Hey, what's that money-pile shaped tarp doing in the parking lot?" would it be her fault or the bank's fault when the money went missing? That's what just happened here.
  • adria.richards
    @Bob,
    Everyone can make choices on what they think about other people.
  • adria.richards
    @TJSwift,
    I am surprised at what 5 year old's can do. I've seen them easily pull up websites, navigate and remember where to go.

    They may not be able to read all that good or use email but they can click and that's all this took.

    My specific example for a 5 year old is if his older brother was already at the Norm Coleman site, the 5 year old could have downloaded the databse.

    A fifth grader would be capable of finding the IP address, going to the site and downloading the file.

    Children have a high capacity for learning that I think most people underestimate.

    Nine-Year-Old Prodigy Writes iPhone App
    Teaching Children and Kids to Program the Old School Way - Scott Hanselman's Blog, Microsoft developer with kids

    and I'm trying to think of the name of the 14 year old website programmer who has been recognized in the industry as an up and comer in the field.
  • Greg
    @Bob, Bob, who are "you people"? And your bank analogy is inaccurate: banks have a duty to take appropriate steps to safeguard their depositors' money. This means they do not, for example, leave the money in a shoebox by the side of the road. Essentially, by leaving contributors' data in a public area of the website, the Coleman campaign has done something very much like that. Can you show me that the author of this site has done anything actionable by exposing that fact? And if the Franken campaign had made the same error, and had failed to take reasonable steps to secure sensitive contributor data from public view, the error would be just as egregious -- failure to meet a basic standard of care isn't a political issue.
  • Countless others could have done what she did. Now that Coleman is caught with his pants down, don't be blaming the people who busted him.

    You're angry with the wrong person. Blame Coleman.

    Wait, that makes too much sense, so you won't do that.
  • Who are "You people"? If you're going to talk about a lack of ethics, better look at Coleman's campaign people who didn't properly protect their donors' data.

    Typical neocon, trying to blame liberals for your own party's misdeeds.

    Here's a shoe horn to get your granny panties unbunched.
  • adria.richards
    @annie,
    You are welcome
  • Michael
    @Bob, Wow Bob you really are an idiot. She didn't take anything she didn't break in. All she did was point out to Coleman that someone could easily break in.

    Not really a crime, Now Norm doing nothing about is actually a crime
  • Bob
    @Angela, I guess a bank robber's criminal defense should be "obviously the bank didn't take the necessary precautions to protect their customers' money." You people have never been much on ethics, though, have you.
  • Bob
    @Dave, Yeah, blame the shopkeeper for the brick through his window.
  • Bob
    It's too bad your technical training didn't include any lessons on ethics. You're no better than a common thief.
  • DMan
    @Dean W.,
    Nice little troll. Have any good ideas lately?

    Where is it exactly that she posted the database commander cloud head? I read the darned post 3 TIMES and could not find it. Wanna help me out here?

    Try sticking to what you understand, which is...........
    Never Mind.
  • TJSwift
    This was more than something "a five year old" could do, and you know it. I don't know any five year olds that go around "pinging" for IP address.

    You were deliberately poking around where you knew, or certainly should have known you were not supposed to be. You could have contacted the website administrators and informed them of their error, but instead, you then made a conscious decision to put the finances of hundreds of people who have never done you any harm, in jeopardy.

    Your actions, excuses, and complete lack of conscience, as well as the reaction of Franken's supporters speak volumes about just how low the left has sunk.

    I'm no lawyer, but I hope there is a really mad one among the people whose identities you've put at risk.
  • annie
    hey, thank you so much for making this extremely easy for a person who knows nothing about computers to understand.

    how timely. life works in mysterious ways.
  • Dave
    @Dean W: Your nutty post reassured me that Franken is the safe and sane choice in this contest. So I'm going to alfranken.com right now to give him $100 using my debit card. I trust Franken to respect my privacy and legislate honorably and nobody can honestly say the same of Coleman.

    Coleman's campaign negligently published their donors' private information on their website and criminally failed to notify the donors when that fact was exposed. Now the campaign is wasting law enforcement resources on a shameful attempt to blame the whistle-blowers. This only came to light because of their botched attempt to get attention by faking a website crash. None of this behavior is acceptable.
  • I think you did an awesome thing. Why do some people have their panties all bunched up over this? You did nothing wrong. It's COLEMAN'S CAMPAIGN that erred here, in not taking people's private financial data seriously. I think that says a lot about him, and I'm glad his days as a feeding-at-the-public-trough politician are numbered!

    Thanks for posting all of this information. I found it fascinating, and your motivations are honorable.

    Keep up the great work! I hope this gets you some paying gigs!

    Too bad the TV news didn’t do a better job of reporting on this. They basically let the Coleman camp write the press release and took it at face value.

    Excellent job!!! Thanks for going public with what you know and what you found.
  • Adria Richards
    @Richard Wright,
    Agreed. There seems to be a lot of denial in the technology communications from the Norm Coleman office and that's what tipped me off to look further into it.

    I agree, taking the site down for a few days would have been the best route. By saying the Feds went through the log files and "confirmed" no one hacked it has nothing to do with if it was setup properly or that it was being monitored.

    Yep, I know there will be Trolls who stop by and leave fake email addresses with their comments. I'm certainly willing to post them as long as they remain respectful.

    Thanks for the support! I appreciate the words of encouragement. The goal is to educate and inform those who do want to understand this issue technically.

    "shoot the messenger" doesn't stop the message folks.
  • Richard Wright
    Adria: well done.

    The indictment here isn't of human fallibility but political dishonesty.

    Knowing full well that something was amiss, the Coleman campaign people should have pulled the site and done everything to discover the cause of the problem.

    But they didn't.

    The temptation to make political capital was too great -- to spin an IT problem into a success story by claiming hey, look at us, we're so popular / we're thought to be so influential that thousands of people flock to our website.

    On which basis, were these people in charge of a school bus heading out of a control down a steep grade, they'd be saying nothing about the brake failure and everything about how their engine was far more powerful than anyone else's: wheee! Look at us! We're the fastest anywhere!

    Politics rarely admits of ethics (hence the phony posts you've earlier received from individuals whose "criticisms" are actually Coleman damage limitation exercises) so you were fully justified in everything you did to expose the truth of what happened here.

    Thanks, to you and all those others for their vigilance.
  • Adria Richards
    @24AheadDotCom,
    Everyone is entitled to their own opinion. Each person controls their actions and that's where most people fail.

    Norm Coleman's office had the opportunity to be transparent about this problem, acknowledge something was wrong with their site and bring it offline. Instead they pointed the finger to politics causing their technology problems (high visitor traffic because voters were upset).

    At anytime, they could have contacted a well know, local, website hosting provider...like VISI.com and said, "Look, we're having a major problem here. Could we borrow one of your senior level guys to take a look at this? Could you keep this confidential?". But they didn't.

    Instead, they claim they brought in the Feds, Secret Service and who knows what other Federal agencies to "look" at log files. All they needed was a really good Linux Admin (I call them Uber Admins because Linux is hard!)

    Many people in the world do a lot of talking (judging, blaming, criticizing) but most won't do the walking (action, progress, forgiveness).

    It's a logical fallacy to base the quality of my work on this single post.

    This problem may have occurred because the Norm Coleman website guy was too proud to ask for help or he was not experienced in:
    1. Handing a server attack
    2. Handling a website attack
    3. Monitoring a website during transition
    4. Maintaining a secure website
    5. Setting up security notifications
    6. Reading log files
    7. Monitoring DNS
    and so on.
  • I recently discovered what looked like a website hack, where some JS to load what was presumably a virus-laden PDF had apparently been put in a site's sidebar.

    Did I post about that on the web? No, I sent emails to the site owner and to their host, and then I followed up.

    No one should hire someone who'd write a post like this, because you can't trust her to do the right thing.
  • Adria Richards
    @Dean W.,

    I'm doing great. I'm blessed with a good heart, information about technology and the desire to help people.

    The Emperor has no clothes. I'm telling grown up men their site isn't setup right.

    Do you really think if I'd called the Norm Coleman office at 7pm that night, anyone would have answered?

    After the information was posted, they had another chance to resolve it and address it.

    It seems that since they did not contact donors, other people, who had downloaded the database felt they needed to notify these people.

    In the video at <a href="http://askadria.com/2009/03/11/norm-colemans-database-revisited-and-website-development-choices/" rel="nofollow">AskAdria.com - Norm Coleman's Database Revisited, I share that previous attempts at contacted businesses about their security problems have failed. I decided to document how I found the information, what it means and how other companies can protect themselves from this security problem.

    I sleep well at night.
  • Dean W.
    Can I ask you a question?

    What the $!%#$ is wrong with you?

    You discovered a serious security deficiency; so what did you do in response? Did you attempt to discreetly contact the website to warn them about the problem? Was your first concern to save these accounts from potentially being victimized? Apparently not.

    No--YOU POSTED THE $!%#$ INFO ON THE WEB--so that any criminal could go ahead and victimize these accounts. Is this your idea of being a "Good Samaritan"?

    Again, WHAT IS YOUR PROBLEM???

    Are you Democrats so self-righteousness that you feel you are morally licensed to abet criminal action against those who disagree with you?

    Are you totally devoid of even the most basic sense of decency?

    Apparently.
  • bbuc
    Trying to Drag Us Back to Stupid

    Repulican Coleman and his staff's general incompetence and whiney, smarmy behavior remind me of the Bush years all over again... and just when I was starting to get over it.

    God I'm glad they're gone.

    What a feckless putz he is.
  • asdjijdijd
    YES! Long Live Liberty! wake up sheeple! good work man
blog comments powered by Disqus
top