Health knowledge made personal
Join this community!
› Share page:
Go
Search posts:

Complexities in Data Systems Growing Beyond Control –“Algo Duping” Society Combined With A World of Rogue Algo

Posted Aug 03 2012 11:58am

As I’m sure everyone is aware of what has occurred with Knight Industries with more than $400 million wiped out by an algorithm…they do have teeth as I have written here so many times before.  In addition you can read in the article clips below that they were running “new software” to work with the NYSE dark pools that just came online.  Problem was, the software didn’t work with the new “code” that the NYSE had created.  There’s a ton of time that goes into producing software and the average consumer “has no clue” on how complex this is and it is difficult to understand unless you have spent any time writing software yourself.  Programmers and developers know it no matter what language they work with. 

The problem with new code and updates is not restricted to stock markets by any means but of course it’s where money is made and lost and is the foremost area of concern as economies are based on what happens with the algorithms in that environment and other industries want performance but not near to the speed that is pushed to the limit on Wall Street.  I have been saying this for quite a while now that it is time to clean up the math and coding in all industries. In the words of Bill Gates, software is nothing more than a bunch of algorithms interacting with each other, so think software, think algorithms. 

We live with this all the time with code going south, just look at the social networks and of late they all have had their issues but it’s not money and transactions for the most part, Twitter, Facebook, LinkedIn have all had issues of some sort and when they come back up or fix a security breach it’s back to business as normal.  This is the hi tech industry folks and it touches more than the markets.  The most glaring issue of course that is still not resolved was the Facebook IPO which basically was out there to not only make a bunch of people richer, but to convince the public on the over inflated values of “social algorithms” and you can see by the price and what has occurred, it’s not there.  When you invest in technology, what do you have, a piece of an algorithm?  That’s really the bottom line as they are pretty much the intangible businesses that function out there today if they are not producing some kind of hardware along the line. 


There’s also nothing wrong with being an “algorithm shareholder” either as long as the value is appropriately stated and the code runs correctly, but of recent in many instances we are not seeing that.  Algorithms move money and have teeth as well as make life impacting decisions about all of us.  The amount of flawed data being collected is also growing when credible data is combined with non credible data and there’s no problem with crunching numbers to enable some predictive analytics, that is until someone want to sell some software and takes it down to an individual level to “score” people with credible and non credible data combined…and then we get “The Attack of the Killer Algorithms” as flawed data is the result.  It happens all around us and basically I stated a while back that this methodology of “duping” naïve consumers is what created the “Occupy” movement. 

The movement and people involved I don’t think even realized what real root of all of this is, but they knew something was wrong.  It’s a hard battle when formulas run on servers 24/7 and you can’t see the algorithms, talk to them or in many cases get a customer service person who can adequately explain the decision making process that companies are using today, and frustration sets in.  I wrote a series of blog posts that talk about algorithms in every day life, be it a health insurance claim denial or someone not getting a job, it lives amongst us.


“It is a technological arms race in financial markets and the regulators are a bit caught unaware of how quickly the technology has evolved”

Back about 3 years years ago I made an “opinion” post asking if “we need a Department of Algorithms”…maybe we are there as there are those algos created for “desired” results and those that generate “accurate” results and the two should be the same, but they are not always.  Read this book on how some of this works from Twitter bud Professor Siefe at NYU, as he nails it pretty well.  We are well beyond the comprehension levels of those who make our laws as well and you saw what happened yesterday when they don’t get it, they go on break and leave everything unattended not even caring about the Post Office crisis. 


Here’s a couple other past posts here that may be of interest that are “on topic”. 

I thought I was the only one talking about this but recently was contacted by the National Institute of Statistical Sciences that told me to keep blowing the horn and that someone will eventually listen.  It’s all about the math, algorithms, data, and “flawed data” that creates markets sometimes where they don’t belong. 
image


Do keep in mind too that the amount of data selling by some of these technology companies is growing at a speed that is even outpacing Facebook with mining data on the web where they get their “data for nothing and the profits for free”..so that goes on as well behind the scenes where consumer data is mined and sold, in other us as consumers become part of the profit.  This is a good time to remember that health insurance companies ride this wild roller coaster on the market too and again the question of having to pay one of these companies for healthcare comes up time after time and again goes back to insurance being a “non profit” instead of what we see out there today. 

We also have marketing like we have never seen before and sometimes we have a train wreck there as well.  Marketing and data people can push the developers to release software before it’s time and it happens.  Look at the Knight story again for proof and this happens everywhere, not just markets.  One interesting story was the big fiasco with Allscripts who projected sales based on projections of where they would be with combining two technologies and I flat out said, too much code too little time.  Again we come around to those who sell and market flat out just don’t get it with today’s complexities.  That post is Chapter 30 in the Killer Algorithm series.  It’s not popular to talk about this and most do not as keeping the level of comprehension gray helps sell markets but eventually it all comes out.  Interesting too with the Knight incident, we are now hearing the word “algorithm” use more with the media instead of the ridiculous “circuit breaker” terminology.  Call it what it is. 

Amazing how some of this works and how profits are derived and again I have said many times that we are to the point to where data selling should have an excise tax as it is part of every day trading business as well as other industries, and you take infamous example of Walgreens in 2010 making just under $800 million on selling data only?  There’s a pot of gold out there just waiting to be taxed, but again we don’t have lawmakers with enough technology backgrounds to see this and the further behind we fall.   The algorithms live amongst us and here’s another recent post I made (and thanks to the folks at Nielsen for their kudos on this one) which breaks some of this down a little further.  Keep in mind here too that I am making an effort to take some pretty complicated issues and bring them to a level to where the layman can get something out of this as well as I don’t anybody else out there doing a lot of this.



It really gets interesting when a source of “flawed” data is challenged and we want answers.  What happens?  When the data and media spin it to the point of being out of belief, those folks go back and create more formulas and reports to substantiate the past and some of it does get down the the truth and some of it is just a further “spinning” of information to substantiate the original questions and of course this is the hot topic when formulas and algorithms have either denied consumers or made big profits on conclusions of data queried that are not accurate.  I think we all know too that journalists have to meet certain ratings to keep their jobs so again not to fault some of the reporting out there as they need their jobs too but does this impact what we get out there, sure it does and if I had to make sure I kept my job I would probably be doing the same thing. 

Sometimes so much of the intelligence we rely on today get’s spun and again there are folks out there willing to write code for money no matter what it is and you kind of get some of this mentality mixed in with the good productive algorithms we need and again this mostly applies to the erroneous reports where credible is combined with noncredible but it’s not exclusive either:

”Hey dude let’s crunch some numbers and see if we can come up with some analytics to sell”

Yesterday there was an article out about how it’s going to become complicated for doctors to figure out which women covered by which policies will be entitles to the free no co-pay birth control
and I commented on Twitter, who’s going to be out there with a new algorithm to help doctors figure it out?  This is how it works with complexities it builds on itself and I guess I can keep my eyes open for a press release for some new software or existing software company talking about their new algorithms to sell that will make it easier for doctors to determine what women do get the no co-pay birth control, complexities out of control  We absolutely put consumers on a data chase at a time when they just want care.  Killers Algorithms Chapter 6 talks about this to where the data has already made you guilty due to a flawed system at times, so let’s see if this happens?  Will we have a “free birth control algorithm” next?

Here’s yet another interesting radio interview that offers a lot of good information, simple enough for the layman that again addresses this issue with spun data and marketing, further continuance of “algo duping” the consumer today. 

Run no code before it’s time…yes and Nanex yesterday was out there with suggesting better testing of software in the markets but when there’s money to be made, well you get the picture there.  Again, this is a dangerous area to where the rush for greed and the ability to not only control a market but also a society is upon us and you can’t see the algorithms, talk to them and as we saw yesterday with flaws they do their thing.  A couple weeks ago too someone wrote an interesting article about the real time updating of prices at Amazon that could signal a danger area as well with the potential of a “flash crash” type of incident there and it was good food for thought as again we are talking masses of data and algorithms. 

You may or may not have heard about their infamous rogue imagealgorithm that priced a copy of a book about flies at over a million dollars, so again it lives out there with us and be aware as the Attack of the Killer Algorithms, no matter where you are will show up when you might least expect it.  The video below includes this story in How Algorithms Shape our Lives…if you not have not seen it, a must see that will help explain some of what goes on today.  As Robert Scoble mentioned in one of his TED presentations “the world belongs to the geeks today because nobody else wants it” and those who write the code rule the world.  This video by the way has been out there for about a year now and talks about contemporary math and algorithms.   Find out what Black Box investing is…investments as he says “formerly known as your pension”….



So again until we regain some balance with the markets with more tangibles as those companies create jobs, we are stuck with what we have today, formulas written for “desired” results and those written for “accurate” results and they are not the same as they should be and the expertise of the banks and brokerages to keep the game moving, but there will be a point to where this bottoms out to when the cash is no longer there that has been created with some illusionary programmatic algorithms to sell and market.   BD




“We put in a new bit of software the night before because we were getting ready to trade the NYSE’s RLP program. This has nothing to do with the stock exchange. It had to do with our readiness to trade it," Joyce told Bloomberg TV in an exclusive interview.

"Unfortunately, the software had a fairly major bug in it. It sent into the market a ton of orders, all erroneous, so we ended up with a large error position which we had to sort through the balance of the day. It was a software bug, except it happened to be a very large software bug, as soon as we realized what we had we got it out of the code and it is gone now. The code has been restored.

http://www.wallstreetandtech.com/electronic-trading/240004831


Post a comment
Write a comment: