Cathy O’Neil, Mathematician/Quant: Wall Street Quants The Culture, Big Data Mechanics, Algorithms, Data Mining, Lack of Pr
Posted Dec 31 2012 6:10pm
This is pretty good and I felt compelled to include this as she’s the one before the programmers, in other words, Quants build models and programmers design and build it. In both videos you will find a lot of answers as she’s good and if you read here then you have probably heard a lot of this from a programming side. She’s the first step ahead and created the model that programmers use to write software. She has had a very interesting background and she digs in to big data and that subject I think be renamed, the “data scientist”…same thoughts where in the world do you find these people. You can find her blog here, MathBabe, like the name.
She gives her background about being a young nerd and she has been a professor and later was hired by a Wall Street Hedge Fund as a Quant. She worked with Larry Summers who ended up going to work for the Obama Administration. Her idea at the time was to “fix” the system. Nobody including the SEC wanted to hire her and she says the same thing I have said many times, get the attorneys out and get the computer scientists folks in there. She said nobody cared with the models and some of the CDO models were completely broken and gave random numbers. Clients were never given fair warnings she said and she gave up and left and applied again to the Feds and and she signed forms to where she could not give away the private information on the models but she could talk about “techniques” of models and that’s what she does on her blog. Her idea is to help people, not exploit them. She says portfolios are “dummied” down into single numbers so those that think they understand can understand but they don’t really understand:) My thoughts from a short while back on risk.
Models are used increase obscurity and are not explained and are abused. I made my comments on models used out of “context”. Avoid doing harm she states. She still models and knows the benefits when done correctly. Nerds are never in charge. I just talked with someone the other day where a study was done and the data was not supplied to substantiate the abstract and without the data, the recipient determined there were over 500 million available models, data counts and you just can’t take to the bank a report or study you see in the news without offering someone the opportunity to confirm the data origins, etc.
A little later on she talks about how models get used “that was a really useful model to make a lot of money”…you’ve had heard something like that here before:) People are trained to think of mathematics as a game you have to “win”, the culture in finance. No Quants are ever expected to answer or account for the consequences on the effects of normal people. Modeling has gone crazy and it has and it will.
You can speed up to around 18 minutes where the healthcare portion comes in. Think about 15 years from now she says, the information on the web about you never goes away. De-Identified data, she says the same thing I said a few years ago, people are good at matching up De-Identified data, heck I did it years ago and not for the purpose of selling data or anything of the sort, it was some algorithms that I wrote for integrating software and to check and balance myself I created sample data that would rematch in the system for billing and an automated super bill system.
Old hat been around a long time and a couple years ago the OMG stuff on the web about this being possible, that’s just when the public woke up. It’s not that we are really that smart it’s just that we have “been there done that” way before a consumer even hears the concept, why we are geeks:) What is great about her videos is that she does for modeling about the same as I do for programming and there’s nothing like hands on as everyone else just reads other stuff and repeats it. She addresses both sides of healthcare data and says the clinical side is great and of course I agree, but then we have the health insurers who use segmentation in the opposite direction.
She did some other jobs and a lot with credit risks. Listen to her job offer about modeling in Hollywood and she’s see right through it with crazy modeling and how it keeps inequality alive. If you want to hear her points on healthcare move forward to around 16 minutes. Here’s a second video to where she talks at the Hadoop Strata Conference. She says you can’t just rebrand people from the inside and turn them into data scientists.
Here’s the series on the Attack of the Killer Algorithms that go right in line with what Cathy says and watch the videos and they are also on the left hand side of this blog on every page. If you want a look into reality on how math and formulas rule the world, spend the time and get a tiny bit of education to understand how this works quietly behind the scenes and how the algorithms shift and take your money, a few SQL “strings” at a time. BD