Saturday, October 3, 2015

NCEP/NCAR index up 0.06°C in September

The Moyhu NCEP/NCAR index from the reanalysis data was up from 0.306°C to 0.368°C in September. That makes September warmer by a large margin (0.05°C) than anything in that index in recent years. It looked likely to be even warmer, but cooled off a bit at the end.

A similar rise in GISS would bring it to 0.87°C. Putting the NCEP index on the 1951-1980 base (using GISS) would make it 0.95°C. I'd expect something in between. GISS' hottest month anomaly was Jan 2007 at 0.97°C. Hottest September (GISS) was in 2014, at 0.90°C. It was the hottest month of 2014.

The global map shows something unusual - warmth in the US and Eastern Canada. And a huge warm patch in the E Pacific. Mostly cold in Antarctica and Australia, but very warm in E Europe up to the Urals, and in Middle East.

Thursday, October 1, 2015

Optimised gridding for temperature

In a previous post I showed how a grid based on projecting a gridded cube onto a sphere could improve on a lat/lon grid, with a much less extreme singularity, and sensible neighbor relations between cells, which I could use for diffusion infilling. Victor Venema suggested that an icosahedron would be better. That is because when you project a face onto the sphere, element distortion gets worse away from the center, and icosahedron faces projected have 2/5 the area of cubes.

I have been revising my thinking and coding to have enough generality to make icosahedrons easy. But I also thought of a way to fix most of the distortion in a cube mapping. But first I'll just review why we want that uniformity.

Grid criteria

The main reason why uniformity is good is that the error in integrating is determined by the largest cells. So with size variation, you need more cells in total. This becomes more significant with using a grid for integration of scattered points, because we expect that there is an optimum size. Too big and you have to worry about sample distribution within a cell; too small and there are too many empty cells. Even though I'm not sure where the optimum is, it's clear that you need reasonable uniformity to implement such an optimum.

I wrote a while ago about tesselation that created equal area cells, but did not have the grid aspect of each cell exactly adjoining four others. This is not so useful for my diffusion infill, where I need to recognise neighbors. That also creates sensitivity to uniformity, since stepping forward (in diffusion) should spread over equal distances.

Optimised grid

I'll jump ahead at this stage to show the new grid. I'll explain below the fold how it is derived and of course, there will be a WebGL display. Both grids are based on a similarly placed cube. The left is the direct projection; you can see better detail in the previous post. Top row is just the geometry (16x16), the bottom shows the effect of varying data (as before 24x24, April 2015 TempLS). I've kept the coloring convention of s different checkerboard on each face, with drab colors for empty cells, and white lines showing neighbor connections that re-weight for empty cells.

The right is the same with the new mapping. You can see that near the cube corner, SW, in the left pic the cells get small, and a lot become empty. IOn the right, the corner cells actually have larger area than in the face centre, and there is a minimum size in between. Area is between +-15% of the center value. In the old grid, corner cells were about 20% area relative to central. So there are no longer a lot of empty cells near the corner. Instead, there are a few more in the interior (where cell size is minimum).

In that previous post, I showed a table of discrepancies in integrating a set of spherical harmonics over the irregularly distributed stations:
L 12345
Full grid00001e-06
Infilled grid8.8e-050.000290.0010450.0020150.003635
No infill0.0076320.0273350.0493270.0644930.075291

In the new grid, the corresponding results are:
L 12345
Full grid00001e-06
Infilled grid8e-060.000190.0006450.0013480.002529
No infill0.0047580.025580.0477940.06010.069348

Simply integrating the SH on the grid (top row) works very well in either. Just omitting the empty cells (bottom row), the new grid gives a modest improvement. But for the case of interest, with the infilling scheme, the result is considerably better than with the old grid.

Friday, September 25, 2015

GWPF wimps out

In April, there was a big story summarized by a headline in the Telegraph: "Top Scientists Start To Examine Fiddled Global Warming Figures" I wrote about it here. The GWPF announced an inquiry into global temperature adjustments. They had assembled a panel of apparently reasonably well qualified scientists. They promulgated terms of reference (which seemed to borrow heavily from the terminology of Paul Homewood). They put out a call for submissions, with a deadline of 30 June. They said "After review by the panel, all submissions will be published and can be examined and commented upon by anyone who is interested." and set up a page for this purpose here. All this promptly echoed, eg here and here.

I immediately thought about putting in a submission, and did in fact write one. I sent it to the prescribed email address on 2 June. No response. So I emailed again to ask if it had been received, on 14 June. Still nothing. So then I wrote to the GWPF general email, and got a prompt and courteous response from none other than Andrew Montford. He said he couldn't find my submission there, so I sent him a copy, which he received. Encouraging.

Still no response from the actual panel though. I kept an eye on the site, especially the submissions page. I thought they might say that they had received (with thanks) x submissions, or some such. But AFAICS, the site didn't change at all.

There was one link on the page that said "news", which was an obvious place to try, but it didn't seem to connect to anything. Three months later, still wondering, I got a helpful direct link from a correspondent, to this page. And there I find, dated July 22, this information:
"The team has decided that its principal output will be peer-reviewed papers rather than a report.
Further announcements will follow in due course." report! So what happens to the terms of reference? The submissions? How do they interact with "peer-reviewed papers"?

And of course one may ask who (if anyone) will ever write those papers? And about what?

I wonder what changed their minds?

BTW, here is a Wayback snapshot from June. I don't think anything has changed, except for the "news".

Tuesday, September 22, 2015

Better gridding for global temperature

Computing global temperature is an exercise in spatial integration with scattered data. I have written a lot about it previously, eg here or earlier here. A spatial integral is a weighted average, so it comes down to calculating the weights. With TempLS, I first used a grid method, as is traditional. Then, to overcome the problem of empty cells, I used an irregular triangular mesh, as in finite element integration. I have also developed, and will soon describe, a method using spherical harmonics. I think the later methods are better. But grids also have some advantages, and I have long wanted to get a rational infilling basis.

Numerical integrtaion

Integration is usually defined as a limiting process, whereby the region is subdivided into finer and finer regions, which can then each be evaluated with some local estimate of the integrand. There is theory about whether that converges. With a finite amount of numerical data, you can't go to a limit. But the same idea applies. You can subdivide until you get a result that seems to depend little on changing the subdivision. Sometimes that won't happen before you run out of data to meaningfully estimate the many subdivisions. That's one reason why temperature anomalies are important. With absolute temperature, you would have to divide very finely indeed to be independent of topographic variation, and there just aren't enpigh reading locations to do that. But anomalies take out a lot of that variation, making practical convergence possible.

You might ask - why bother with different methods, rather than finding just one good one? The answer is with this idea of reaching an invariant zone. If we can find an integral estimate that agrees over several different methods, that will give the greatest confidence in the result.

Grid considerations

With gridding, you can choose a coarse grid so that every cell has data. But then the data may not be a good estimate of the whole grid area. You lose resolution. A finer grid will start to have cells with no data. Traditionally, these are just omitted, meaning in effect that they are assumed to have the global average value. This was improved by Cowtan and Way 2013, using kriging. I proposed a simpler approach using latitude band averaging, which gave some of the same benefit. In this post I'll look at upgrading the infill process, using numnerics similar to solving the diffusion equation. It tries to find a local average to use for each missing cell.

Improving on lat/lon grids

To do this, I need a better grid system than lat/lon. That creates a big problem at the poles, where cells become very small and skewed. The essential requirement of a grid is that you can quickly allocate a set of scattered data to the right cells, and you know the area of the cells. There are many other ways of doing this. Lat/lon is based on gridding the sphere as if it were a flat surface, which it very much isn't. You can do much better using a somewhat similar 3D object. Regular platonic polyhedra are attractive, and an icosahedron would be best of these. But a cube is more familiar, and good enough. That is what I'll use here. The cube is gridded in the normal way, with a square grid on each face. The sphere surface is radially projected onto the cube.

I'll give details, with the infill process, and tests of the improvement of the results, using spherical harmonics, below the fold. And of course there will be the usual WebGL active picture. It will show the cube grid projected on the sphere, and infill for a typical month, with lines to show the infill dependency.

Thursday, September 17, 2015

Land Sea interface in global temperature averaging

Inhomogeneity is a problem when estimating an average. You have to sample carefully. In political polling, for example, men and women tend to think differently. So you need to get the proportions right in the sample (or re-weight).

In a global temperature average, a big inhomogeneity is the land/SST difference. For grid-based estimates, a land mask is often used. This tells how much of each cell is land and how much sea.

I haven't used a land mask with TempLS grid, because I think grid weighting has bigger problems. And with mesh weighting, there isn't any clear way to mask, especially as there is typically a new mesh for each monthly set of stations.

The mitigation is that there is a lot of cancelling effects. Some land areas may be in effect represented by sea, but also vice versa. Island temperatures tends to influence surrounding sea, but then again, they really should. If they were only representative of their own land area, it would generally not be worth including them.

I may still try to do something more elaborate. But in the meantime, I thought I would test what the current algorithm does, using what maths call a color function. This is 1 (red) for land stations, 0 (blue) for SST. I plot it as if it were temperature. I hope to see that land areas uniformly red, sea blue, and an in-between color tracking the shore. Insofar as it fails to track, I hope the failure is balanced, so that neither sea nor land is over represented on average. The result is an active WebGL plot below the fold.

New sea ice treatment in TempLS

TempLS is my global temperature anomaly calculator. It uses ERSST V4 for the sea surface component. ERSST unhelpfully records sea covered with ice as having the sea water freezing point of -1.8°C. Since this does not seem to be a good proxy for air temperature, I mark such points as missing data.

However, it seems -1.8 is not reliable - regions which are clearly frozen can report higher temperatures. Maybe there is an allowance for salinity variation. Anyway, to date I have been using ≤-1°C as the criterion for removal.

I now find that helps, but also doesn't work reliably. Even with the Arctic ice near peak, there seem to be a few deviant points reporting SST where there should be none. This creates a problem with my triangular mesh weighting in the Arctic. The idea is that land stations should be a better basis (than -1.8) for estimating nearby frozen seas. But these isolated spurious sea temperatures also become prominent in the mesh, and affect large areas, as if they were land.

So I am using a new criterion. For each SST location and month, I look at the record starting 1900, and count the occurrences of temperatures less than -1.5°C. If there are more than 10, I deem the location to be subject to intermittent freezing, and exclude it for that month throughout.

This will clearly exclude some valid data on the fringes of the ice. However, with some years frozen, others not, it is in any case hard to get an appropriate normal. In practice the decision about inclusion does not have dramatic effects for individual locations, since freezing leads to a zero anomaly. And because it is a lat/lon mesh, there is an artificially high node density anyway.

You may notice some small differences in the TempLS mesh results. The August average rose from 0.7°C to 0.703 °C.

Tuesday, September 15, 2015

Temperature change July to August mapped in GISS and TempLS

I was curious about the relatively small rise in the GISS August global land/ocean anomaly, relative to July. So I drew maps comparing gridded GISS with TempLS mesh for those months. The maps are based on a spherical harmonics fit to both data sets, with L=10 (121 functions). This puts them on exactly the same basis.

I included TempLS with GHCN adjusted data as well. This gives the opportunity to also show the differences between adjusted and unadjusted, and to plot the actual differences.

Nothing looked unreasonable. The big Jul-Aug difference between GISS and TempLS was in the treatment of Arctic (especially) and Antarctic. There is actually very little effect of GHCN adjustment. The month-month changes are themselves quite interesting; the patterns are quite persistent.