“How difficult is it for a motivated scientist, well versed in programming-probability-statistics, to build an automated trading system from home?”
I’ve done this, and I’m not a scientist (I’m an ex-con who spent time in prison and wanted to change his life). I made plenty of mistakes along the way, and would not proceed as I did starting again.
My friend and partner at the time (who runs the excellent blog Bent on market domination) and I used C# to automate several systems I had been trading for several years on Ninjatrader, hooked to Interactive Brokers for execution. I was already a competent trader with a strong background in trading systems development. Ninjatrader is garbage software, objectively. I chose it because it was cheap, the data was cheap, it hooked well to IB, I would absolutely not have anything to do with it again (though I highly recommend interactive brokers for the automated trader). Either build your own system (Andreas Clenow has some excellent info on his blog) or use Quantopian (in python) or Quantconnect (in C#)
Retail level programs like ninjatrader, multicharts and tradestation have many many problems for the fully automated trader, which are virtually dealbreakers long term. You are absolutely going to spend as much time monitoring your software and data feed as you would just trading that model manually. The biggest problem with Ninjatrader (aside from reliability of data which is terrible) was that the backtester was just a complete work of fiction. We had to write our own tick data backtester, which ran incredibly slowly. Also, none of the retail level programs do portfolio level backtesting, which is a dealbreaker for serious trend following programs for example.
The problem for the programmer attempting to become an automated trader is that most often the programmer is attempting to automate a task he does not yet know how to do (trade). “I’ll be a quant”, is currently in vogue, but to be a successful quant you need a deep, not superficial, understanding of markets. Also, beginner traders are obsessed with entry rules and setups, which are really not the main game. Professionals look at exits, correlations, portfolio management, etc. And of course, home trading guys like myself cannot possibly even think about competing anywhere the big players are heavily invested, so HFT is out.
My experience is that many people seek full automation as a panacea for the many emotional problems with trading, only to discover those exact same problems are present just as much with automated trading.
For example, you don’t like the feeling of powerlessness when a position goes against you, but you get the exact same feeling of powerlessness when your software insists on taking a setup you don’t like, or which is against your pre existing bias. Neglecting these feelings can send you quite literally insane with worry.
So now, I don’t trade fully automated. I use the software we wrote to identify setups and place the orders, and manage the trades (in terms of targets, stop losses, trailing stops, etc) but I like having a human in the loop. It goes without saying that any decisions to override the models are revisited and discussed later, to see if that was a good idea or not.
Excuse the awful colors, but this is a system running recently on the August gold contract. Bright green bars represent a long setup (the model only goes long in a trending situation, defined simply by moving average A > moving average B, and is looking for a specifically shaped candle as a trigger. The model goes long on a break of the candle high, with a stop at the candle low). Aqua bars after the green bar represent moving the stop to breakeven. A bar outlined in yellow indicates a stopout trade (a loss)
You can see that this model has generated one big winner, a breakeven trade, followed by a loser, another breakeven trade, and a winner which is currently open and in profit. Nothing special, just grinding it out. The actual edge isn’t anything particularly complicated, controversial (hammer candles in an uptrend), or objectively good. The real “white man magic” is how you manage correlated entries, how soon you move to breakeven, how you manage position sizing in drawdown and super performance periods.
So bottom line, I would say that a pure programmer with no experience in markets would not have any chance without a work environment where he was exposed to experienced traders with good ideas, or was willing to put in the work to understand markets at a deep level. There are no courses I have seen that explore this in enough detail.
Because there are billion dollar funds who hire a lot of maths and physics PhD’s with no trading experience it is tempting to think the markets are just a brainpower competition and you can just point the highest IQ at a problem and win. Reality is not like that. An additional problem is that edges found by data mining (and not from fundamental properties of markets) are prone to work extremely well and then disappear forever. This is a non-trivial problem to solve, one possible solution would be to have a portfolio of “edges” and switch them on and off as they stop/start working.
Yes, its is possible, in fact it has never been more possible, to be a self employed automated trader. The problem is that being an automated trader is just like being a non-automated trader with a different, but similar, set of problems to solve. I would absolutely recommend visiting Ernest Chan’s (who is also on Quora) blog Quantitative Trading , without knowing him he strikes me as someone who smells like the real deal.
(This post is part of a series compilation of discussions held in Quora back in the day.)