Forum | DV Indicators

Guest
Welcome Guest, posting in this forum require registration. Username Password Remember me

Forum » Code » Excel » Quantitative Methods for Choosing System Parameters

Pages: [1] 2

Author	Topic: Quantitative Methods for Choosing System Parameters

SeekingBeta

Member
Posts: 3

Quantitative Methods for Choosing System Parameters
on: November 13, 2009, 16:00

I’m curious what other people use to choose between variants of a quantitative system that use can use different parameters to control the model. I’m looking for numerical methods, instead of just eyeballing equity curves. At the moment, I take the annualized CAGR, I add in a multiple of the annualized Sortino Ratio with ~8% MAR, and then I subtract a multiple of the # of trades per year. This emphasizes high overall return, a consistently uptrending equity curve with limited long-term drawdowns, and limits the trades per year. I’m not a big fan of the sharpe ratio, for example, because it penalizes large up years.

Mr_rolfsson

Member
Posts: 3

Re: Quantitative Methods for Choosing System Parameters
on: November 13, 2009, 18:29

Interesting, could you please write the formula?

Ramon

Moderator
Posts: 6

Re: Quantitative Methods for Choosing System Parameters
on: November 14, 2009, 09:03

Interesting concept SeekingBeta. Penalising upside volatility through use of the sharpe ratio remains a questionable practice in my mind too. That said, the alternatives i.e Sortino, Ucler, Sterling Ratios etc each have their own downfalls. I like you approach of combining metrics very much. David has suggested a similar concept, namely the DVR ratio, which is basically the r-squared of the equity curve x the sharpe ratio. So this captures drawdown and straightness of the equity curve. I use this as my primary objective function nowadays. I have some amibroker code for this if people are interested.

I’m interested as to why you penalise this metric as number of trades increase?

good stuff!

Ramon

WTP

Member
Posts: 14

Re: Quantitative Methods for Choosing System Parameters
on: November 15, 2009, 11:57

Ramon

I would be very interested in exploring using the DVR curve with Amibroker; anything your could share on that would be most appreciate. I can give my email if useful.

Kind Regards,
WTP

SeekingBeta

Member
Posts: 3

Re: Quantitative Methods for Choosing System Parameters
on: November 16, 2009, 02:52

I tweak the formula a bit for each system I analyze, but here is the outline:
Calculate the CAGR (Compound Annual Growth Rate) like this: http://www.investopedia.com/terms/c/cagr.asp
Calculate the annualized Sortino Ratio with 8% MAR like this (it is fairly complex): http://thecuriousinvestor.com/2007/11/13/portfolio-performance-excel-file/
Calculate the average number of trades per year

CAGR + (A * SortinoRatio) + (-B * TradesPerYear)

A is just a multiplier to get the sortino ratio to be large enough to have enough (but not too much) of an effect on the CAGR. B is similar, but negative, to penalize for excessive trading. You could use monthly periods instead of annualized, if your backtest is short. You’ll need to pick A/B values that make sense to your personal risk/reward preferences and the max potential CAGR of your strategy, or possibly express the multipliers as a percent of your max potential CAGR.

The reason I penalize for a large number of trades are the normal ones – trading often involves transaction fees, slippage, possibility of trading errors, time/effort, tax paperwork, etc.

I haven’t looked into the DVR metric yet. If it is doing a fit to a linear equity line (or a log fit for a compounding portfolio), it may not be what I want, since that would also penalize for upside deviation. I only want to penalize for downside deviation, most of the time. I don’t really care how strictly straight the equity curve is (that the rate of change is fairly constant), but care whether it is trending up a large percentage of the time, even if the rate of change fluctuates. I don’t want to penalize periods where the strategy is able to extract large profits from the market. If you are interested in sharing your AmiBroker formula, I’ll have a look and see if it is close to what I’m looking for.

davidvaradi

Administrator
Posts: 29

Re: Quantitative Methods for Choosing System Parameters
on: November 16, 2009, 14:18

hi , you just inspired me to create a new metric DVFE based on the fractal efficiency of the equity curve. perfect efficiency would imply that the curve had no wasted movement, and although short-term profit spikes would be somewhat penalized (nonetheless this is desirable as a profit spike is generally transient and the result of an extreme market regime)–typical upside movement would not be penalized. to calculate this metric one would have to use a continuous equity curve and a fixed bet methodology to avoid a lognormal curve that would distort results.An adjustment is also made for the number of trades

DVFE: CAGRx(absolute value of fractal efficiency)x(1-(1/(the square root of the #of trades))

where fractal efficiency is the net change in the equity curve divided by the sum of the absolute value of daily changes in the equity curve.

cheers
dv

Ramon

Moderator
Posts: 6

Re: Quantitative Methods for Choosing System Parameters
on: November 16, 2009, 17:10

Hey everyone, just a quick note to say that I’ve posted the code for the DVR metric in the Amibroker code forum.

cheers

Ramon

twitch

Member
Posts: 1

Re: Quantitative Methods for Choosing System Parameters
on: November 17, 2009, 17:09

Quote from SeekingBeta on November 16, 2009, 02:52
Calculate the annualized Sortino Ratio with 8% MAR like this (it is fairly complex): http://thecuriousinvestor.com/2007/11/13/portfolio-performance-excel-file/

I’m writing a Ruby library to help with various indicators and ratio calculations (mostly just for my own edification on how these things operate) so I used this spreadsheet as a guide for an implementation of the Sortino Ratio. Here’s the code (there’s no Ruby subforum here , hopefully it will make the calculation clearer to the curious. returns is an array (or any Enumerable, really) containing the returns to evaluate, mar is the Minimum Acceptable Return desired.

    def sortino_mar(returns, mar)
        trm = 1.0   # Total return multiple
        ldsum = 0.0 # Sum of the square of all loss deviations
        returns.each do |v|
            trm *= (1.0 + v)
            ld = v - mar
            ldsum += (ld * ld) if ld < 0.0
        end

        #
        # Downside deviation. I don't fully understand
        # why this isn't calculated in the same fashion
        # as a Standard Deviation but in the sample
        # Excel I created this from, it's not.
        dd = Math.sqrt(ldsum / returns.size)

        #
        # Compound return
        cr = trm ** (1.0 / returns.size) - 1.0

        return (cr - mar) / dd
    end

123

Member
Posts: 4

Re: Quantitative Methods for Choosing System Parameters
on: December 30, 2009, 04:49

Hello all,

First, let me wish you all Merry Christmas & Happy New Year.

Second, I would like to thank you David for an excellent blog and very interesting forum. Also, I would like to thank Woodshedder for working hard and contributing a lot at ibankcoin.com. Last, but not the least, I would like to thank to all posters on mentioned blogs and this forum for sharing their thoughts and opinions.

Before we go into some details, I just wanted to share that I am paper trading several “systems” (each using one of the following: EMA crossover, EMA or MA penetration, price channels) on several instruments (index and commodities futures) and on time-frames ranging over 5min-15min-30min-EOD spectrum.

For all mentioned systems, t-formula has value of 1.38-3.83 and sample size I have in Excel ranges from 150-300 trades for EOD systems and from 300-700 for 5min-15min-30min systems.

t = square root (n) * (average trade / standard deviation of trades)

When it comes to system health check and position sizing I am monitoring several parameters and haven’t settled yet on any of them:

- Expectancy Ratio after each trade and its relations to -1 or +1 / -2 or +2 Standard deviations of Expectancy Ratio after each trade.

- Distribution and visual interpretation after each trade of n rolling trades’ P/L and number of wins.

Please note that f-formula is generally used for position sizing, while I am looking into tools that might tell me when to switch system ON/OFF or when to increase/decrease exposure, due to ebb/flow state system is in.
For readers who are not familiar with f-formula: f$ = (largest losing trade)/((((1+win/loss ratio)*percent wins) – 1)/win/loss ratio)

I find that on some systems these metrics are working fine by identifying underperforming periods for given system, but in some other cases they simply provide no edge. It is interesting, that in some cases they work great for system A, while none of the ideas work fine for system B, despite system A and system B have very similar or even identical Average W/Average L ratios and very similar W% (within 2-3% difference).

When it comes to DVFE – It really came as fresh idea and I’ve decided to temporarily cross DVR and Sortino based DVR from my to-do list, as DVFF seemed to be metrics that is going to provide answers. After some labor in Excel I’ve realized that DVFE works great in identifying underperformance for some systems, while for some other systems it doesn’t work. So, before I conclude that I am back to square one, I wanted to outline what I was doing:

DVFE: CAGRx(absolute value of fractal efficiency)x(1-(1/(the square root of the #of trades))

I took 20-trade lookback and I did calculate after every trade, by using last 20 trades and related values, the following:

CAGR from starting point (trade 50 of 20-trade lookback) till final value (trade 69 of 20-trade lookback)
ABS value of fractal efficiency for same 20-trade lookback period.

After that, Excel calculated FE after each trade, using 20-trade lookback period.

Given FE value in X column I would PERCENTRANK*100 for last 20 values (=PERCENTRANK($X50:$X69,$X69)*100) and if such value, as an exmaple, on trade 69 drops below value of 50, I wound consider that from trade 70 we have underperforming period. If, as an example, on trade 75 we get reading larger than 50, I would consider that from trade 76 we are back into “normal” performance period.

All in all – Do I understand this and am I doing it right when it comes to DVFE?

At the end, if Woodshedder is reading this – I really liked your posts (one of them David outlined on his blog):

http://ibankcoin.com/woodshedderblog/2009/10/18/power-dip-system-health-week-in-review/

http://ibankcoin.com/woodshedderblog/2009/11/01/is-your-stock-trading-system-sick-take-it-to-the-doctor/

Could you please share any ideas how those charts could be used to either Switch ON/OFF systems or to adjust position sizing – i.e. when to be aggressive and when to reduce number of contracts traded.

Thank you all.

davidvaradi

Administrator
Posts: 29

Re: Quantitative Methods for Choosing System Parameters
on: December 30, 2009, 20:16

well 123, that is an excellent post one well worth reading for just about anyone in the community. I think an issue is that you are posing the ,000,000 question that few know how to solve, and the ones that do would surely not want to give away the answer for free. So as a member of the latter camp, I will try to meet you halfway here.

one of the most important factors is not the math or metric used but rather the time frame you are evaluating. For most swing systems, 3 years was the bare minimum to isolate you from noise. More time was desirable, however 10 years was often too much for a simple system. If you assume that a swing system has 125 trades a year and requires three years then you will probably require at least 375 trades to make an evaluation—futhermore if you look at systems with comparatively low t-scores such as those that are intraday (because of program and high frequency trading) you will need 500-600 trades to “turn a system off.” so there is probably issue #1, issue #2 pertains to when to exit and you will find that the best way to exit is when the 500-600 trade average turns negative—so you cannot expect sharp exits, the only solution towards minimizing drawdowns (though it will cost you in terms of returns) is to gradually reduce exposure so perhaps 10% taken off if 125 trade batch is negative, 20% if 250 trades are negative, 30% at 375 (ie you have taken off 60% of exposure by this point cumulatively), and finall 40% at the 500-600 point.

next is the character of the system, the equity curve is simply the derivative of the system indicator itself. If the system is mean-reversion then the curve will be more mean-reverting also. This means that you will have more false signals. In this case you should use a long moving average or slope as in above to turn the system off, and actually wait for the system to underperform before entries to improve risk-adjusted performance. If the system is a trend based system it is easier to trade using a trend based methodology–an example would be switching between relative strength strategies. Short time frames work very well here.

hopefully this inspires some thought…..no doubt i have given away a lot for free here. my gift to you and the community. thanks for the kind words.

happy new year
david

Pages: [1] 2

Forum Server by: Vast HTML, Version: 1.3
Page loaded in: 0.342 seconds.

Posting code in forum: Place any code between the code tags (using this button: ) and place a space after any < symbol, otherwise it will be read as HTML and most likely cut off the rest of the post. Or code can be uploaded to the DV Indicators Google Group and linked to in the forum post (example).

Adaptive. Robust. Optimal.

Guest