PM Daily Market Commentary – 9/2/2015
Gold fell -6.10 to 1133.20 on moderate volume, while silver rose +0.09 to 14.66 on moderate volume also. Gold steadily sold off for most of the day, while silver followed copper and oil in a brisk rally that started prior to the NY open, and after some early selling, continued through to end of day.
The buck was strong today, which may have accounted in part for why gold did poorly.
Looking at the chart, we see gold's failed rally resulted in more selling today. While gold remains above its 50 MA, and that's a positive sign, it has not been able to move appreciably higher over the past week or so. The gold promoters at KWN imagine its ready to burst loose from its chains any day now, but in looking at this chart, I just don't see any evidence of this at all.
While silver still is having problems retaining its gains into the close, it did manage to close higher today alongside its cousins in the commodity complex. Copper was up +1.72%, oil up +4.21%, and just the way silver behaved intraday, it was definitely affected by these positive commodity moves. Things look better to me than they did yesterday; the higher close helped.
Miners avoided following gold lower, but mostly because of yet another end-of-day rally which fished the miners out of trouble once more. GDX closed down -0.29% on light volume, while GDXJ dropped -0.36% on moderately light volume. There really doesn't seem to be much of a catalyst to send the miners higher right now, but someone is certainly buying the dips near end of day.
The USD strengthened today, closing up +0.40 to 95.84 and moving back above its 9 EMA. This definitely helped to pressure gold lower today; gold in Euros closed higher, so we can safely say that gold's fall today was a currency artifact.
The recent USD/SPX correlation continued to hold today; alongside the buck, SPX closed up a strong +35.01 [+1.83%] to 1948.66, recovering about 2/3 of its losses from yesterday. VIX dropped -5.31 to 26.09.
Bond ETF TLT fell -0.86% on the day, making a new low and completely erasing yesterdays gains and then some. Bonds rise a little when SPX drops, and bonds fall a lot when SPX rises. That's not a recipe for success.
The CRB (commodity index) sold off hard early, but rallied back closing up +0.40% and printing a bullish-looking hammer candle on the day. Perhaps this commodity rally isn't over yet!
WTIC (oil) rebounded strongly today off yesterday's big sell-off, rising +1.86 [+4.21%] to 46.05 on some very heavy volume. Oil had a large trading range, but ended up recovering a bit less than half of yesterday's losses by the close. The huge volume and large price swings says there is great uncertainty in the market and a lot of money is changing hands. Still, it is reassuring to see that the oil price didn't simply collapse after yesterday's big sell-off. News articles I read suggest that China was seen buying huge amounts of Dubai crude – as in, almost all of it – the speculation is that they are filling up their strategic petroleum reserve. Seems like the right thing to do when oil prices are low. Every time I hear about such things, I wonder why the US doesn't do the same thing? Are we so allergic to buying straw hats in the wintertime?
If I were China, I'd go shopping for commodities and commodity companies right now. With prices down around 2002 levels, seems like they're a bargain that a cash-rich sovereign wealth fund could take advantage of. Long term ROI would probably beat the heck out of buying overpriced stocks on the Shanghai Exchange. Then again, I don't remember anyone putting me in charge.
Only small changes since yesterday: a bit of good news in commodities – they aren't headed for the dumper immediately, which is reassuring if you own silver, and buyers in the mining shares actually appeared again, which is a positive sign. Perhaps it is that Chinese sovereign wealth fund doing the buying, who can say?
Note: If you're reading this and are not yet a member of Peak Prosperity's Gold & Silver Group, please consider joining it now. It's where our active community of precious metals enthusiasts have focused discussions on the developments most likely to impact gold & silver. Simply go here and click the "Join Today" button.
So at the risk of appealing to just one or two of you out there, I figured I'd toss this out to see what you thought. I took crude oil prices, and I tried to see if there were "hidden frequencies" in there somewhere. That is, repeating patterns of price movement that happen at a certain frequency.
To do this, I used a tool called an FFT. What does an FFT do? Here's an example: if you played a chord on a guitar, recorded it, and then gave that digital signal to an FFT, the FFT would be able to extract the frequencies that each string of the guitar was vibrating at. There would be six horizontal lines on the spectrograph, representing the six frequencies of the six different strings.
Instead of music, I gave the FFT price data from oil from 1983-present, and I wanted to see if there were any similar-looking horizontal lines that would indicate a regular pattern of activity at a given frequency (i.e. something happened every 3 days, or 8 days, etc).
Here are my charts; the spectrogram is the lower chart; the upper chart is oil prices 1983-present, which is what I fed to the FFT:
So bottom line, frequencies definitely exist in the market. There is order, hidden order, and to me that's really interesting.
In the spectrogram, each yellow or red narrow horizontal area is evidence of a pattern of repeated (and high energy) price movements at a given frequency (2 days, 3 days, 5 days, etc) that the FFT was able to extract from the daily price data.
Take the area inside the box I marked "Iraq 1". A bunch of red and yellow horizontal areas appears for the period of the war, then vanishes immediately when the war is over. What does this mean?
This spectrogram is evidence that there was a visible fear/greed effect that came into being during the war. It warped the market, but in a repeating/cyclical (and thus predictable) way. If the effects on price didn't repeat at a given frequency, the lines simply wouldn't be there.
If you understand what I'm saying here (and I suspect only a couple of you engineer-types do), imagine the implications. If we could successfully reverse engineer the frequencies that appear when a given significant event occurs, we could then project those frequencies forward (with their summed interference and reinforcement energies) to actually generate the prices that would subsequently appear, during the time that this fear/greed effect remains in operation.
This spectrogram suggests this is possible. There IS hidden order in the markets, especially when major events occur, but that continuous "red" signal along the bottom says there is also lower-level background cycles that are always present. They vary in frequency over time. Its pretty wild stuff. I'm not enough of a signals processing geek to make this work, but I'd just bet that Wall Street has more than a few of them who can do such things.
This, then, is why charts work. We are the market, and we have internal fear/greed cycles we go through that are based on time, and they are not one simple note, they are a complex chord of notes, and the complex interaction of that chord results in the prices we see in the market.
THIS, then is why I say we cannot determine from one news event why something did or did not happen. Stuff happens that doesn't make sense because the frequencies that determine price have complicated patterns of interaction. Unless you know all the frequencies and how they interact, you cannot hope to successfully determine if today's big move was a Plunge Protection Team effort, or just the delayed impact of a fear cycle that started out on August 11th when China devalued its currency. You simply don't have the information.
I got all this from daily price data run through an FFT. I had to de-trend the data and fiddle a bit, but I don't believe I'm the only one to do this.
I'm assuming FFT is fast Fourier transform, but it would be helpful to know what the y axis and colour spectrum units are.
I'm not sure about price prediction, but might be possible would be market warning – i.e. of significant chaotic events, crashes – based on the spectral intensity, albeit you'd perhaps need to more discriminating between the yellow-amber-red divisions. For example, it looks as if the back end of 2014 was relatively quiet, but that 2015 has been a nosier time.
And it would be interesting, too, to see whether there were any correlations between particular frequencies and the gold price! Again, by eye, the higher gold prices in 2012 occurred during a spectrally noisier time.
So I know just enough about FFTs to be able to call the function with arguments that make sense, after some trial and error. My "experience" comes as an undergrad helping my professor create a spectrogram for looking at speech data. My job was simply to make it go faster. This was a relatively large number of years ago so it was a minor miracle for me to get this code to work. I was really hunting for steady-state frequencies I could use for another purpose, but instead I found this organized chaos.
Here's gold – the LBMA PM Fix. Interestingly, the LBMA PM gold spectrogram looks different than the COMEX Gold spectrogram. LBMA's "artifacts" are much more sharply defined, especially for 1980 and 1983.
After playing around, the Y axis appears to be the frequency. The sample rate I used was "2". Here are the arguments I used to make it work:
Fs = 2 # the sampling frequency
ax1 = subplot(211)
plot(times, startValues) # this plots the top graph with price data
ax2 = subplot(212)
xExtent = times, times[-1]
Pxx, freqs, bins, im = plt.specgram(values, NFFT=360, Fs=Fs,
xextent = xExtent, detrend = detrend_none, noverlap=180)
Here's the description of the function "specgram" itself, which contains so much (related math) jargon it's a bit beyond my limited experience: http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.specgram
I think the answer is, the colormap displays the power spectral density, with red being high power, and blue being low power.
(I wrote my own detrending function – so I'm using that prior to call to prepare the values array, rather than using theirs)
Playing with the arguments definitely changes the results. I wish I knew what the "right" values were for my purpose. Any advice – gladly appreciated. This is the one time I wish I knew more about math.
Dave, you spoke of the FFT (fast fourier transform). The FFT is a fast version of the DFT (Discrete fourier transform). More specifically, the DFT, if I remember correctly, is an “np-hard” problem, and the FFT takes the np-hard problem to be solved just as fast as is possible.
Instead of kN^2 operations, it is now kNlog2(N) operations.
That kNlog2N is really integral to another understanding of mine, entropy. Maybe I’ll talk about that another time.
But the FFT is too complicated to think about. To understand the meaning of the DFT, though… that can be done. And since, in the end, they are both the same, then if you understand the one, you’ll understand something about the other.
So… once upon a time, I figured out how to think about the DFT. I posted it on wikibooks here:
Pan down to the “visualization” section. It’ll show you how the DFT gets its frequencies, by using tighter and tighter circles, that can reinforce each other on subsequent passes.
But all this about the FFT graph… *any* signal has a frequency range. And the less it has to do with any one frequency, the more it shows up on EVERY frequency. So those Iraq War echoes through all the frequencies, don’t mean that it had a repeating effect; rather, they represent that the gold spike was a one-off event.
Now, you are right that people try other signal patterns, just to see what they can do. And one of the ways they tried to deal with things like the unique Iraq Spike, was something called wavelets. Wavelets are supposed to allow unique events to show their echoes too.
Regardless, it doesn’t really help us analyze wierd markets like gold. I suspect that if you wanted to analyze gold, equities, stocks, and whatnot, you’d have to make some kind of a genetic selection-based algorithm… but then, in the end, you’d only know what triggers a particular market, not why.
MR, I appreciate your thoughts on FFTs and the echoes on many different frequencies. I definitely needed a picture to understand WTF was going on in your wiki book description of the DFT. Now I have a sense of what it does, if perhaps, not a deep understanding of why it works. For some reason, my brain locks up whenever I see math notation that reminds me of my lower division engineering math courses. I think I stopped liking math after integration.
When I do an FFT of a particular timerange (versus a spectrogram over the course of the entire series) and then I graph that, I see power appearing at different frequencies. As in the chart below. Circles indicate (at least to me) that there is some signal at that frequency. Am I confused? This is what my first approach was, and its why I got interested in the spectrogram to see if there was any larger pattern I could spot. And then I got distracted by events. Python code below.
N = len(values)
T = 1.0 / 800
x = np.linspace(0.0, N*T, N)
yf = scipy.fftpack.fft(values)
xf = np.linspace(0.0, 1.0/(2.0*T), N/2)
fig, ax = plt.subplots()
ax.plot(xf, 2.0/N * np.abs(yf[0:N/2]))
Very interesting, that. I mentioned that the nlog2N shows up as a matehematical limit of how fast you can do the NP hard problems.
Once long ago I was trying to repeatedly compress data, and see just what kind of compression I could get. As I compressed it I also looked at the frequency histograms, and got a curve very much like what you show. In fact, the more I compressed the data…
* the more the histogram matched a perfect curve
* the less I could get out of compression, regardless of what scheme I used
* the closer I got to a compression of nlog2n.
That started me thinking, that the Nlog2N was a matehematical limit, and the curve I was seeing was a blackbody curve of entropy.
We also know it as a bell curve, and it shows us (for example) the distribution of wealth in a heavily decayed society, or the distribution of intelligence of lab rats IN ANY SAMPLE picked for anything other than test results , or whatnot.
So that says to me that your curve there is a basic entropic curve, and the frequency data is random.
Now, I found that one of the best ways to compress the data was to find where my histograms did NOT match the blackbody curve, and generate a compression scheme that helped it reach that blackbody curve better.
So I would hazard a guess that insofar as your curves don’t match the bell curve, you may be able to extract some energy (profit).
So that’s what I’d look for in the FFT: anomolies from a random distribution.
So here's the thing. When I just feed in the raw data, I get nothing useful. Here. This is what I get when I just feed in the raw prices.
The curve I showed you? That's after I de-trended the data with a 50-point moving average. De-trending is powerful stuff, I'm using it in some other stuff I'm doing for spotting cycles just by eye.
So basically – I think there IS a signal in there.