Perfomance files - how do they work?

pitwall.app - Nice one. Haven't seen that one before. Very useful. Thx.

What is RW data? Real world. Apologies, I should have made that clearer.

"1.02 version.... algorithm"

It must take you quite a while to do all that for each mod; but it's definitely a very thorough way of doing things and I found it informative. I'll be looking forward to your PFs for 1998 for sure.

"I can say things could be easier if Magnussen was not replaced by Verstappen. If complexified the programming."

Oh man, it's even worse in the 70s. Not only endless driver changes, but lots of teams entering just a few events (making it very difficult to select just 22 cars) and even constant car number changes in some years and then there are the races where you only have the one fastest race lap time so you have to project some realistic data for the other cars.

I think I was lucky, when I starting making PFs, that I knew I was probably going end up making at least 30 complete season's worth of PFs and that each of those would have about 17 races: meaning that with potentially well over 500 PFs ahead of me I was forced to simplify from the start as my free time was very limited back then. So as I think I mentioned before, as far as possible I wanted to create something where I could just throw in all the data for each year and (nearly) automatically end up with a set of PFs that would lead to an engaging driving experience and a season of believable results, without me spending too much time on them. Even that process has taken me about five years, but I'm just about at the stage now where I've finished with the endless tweaking! There are very few shortcuts in GP4.

Your method of doing things is truly comprehensive and I have to admire that very much, but such a level of complexity was not something I could have taken on - so I'm actually kind of glad that the data available to me was pretty restricted in the older seasons!

"I decided to take real bhp values from 1998 and compensated them with according grip. In most cases, grip values are between 14000-15200, so it's still drivable."

For sure. While it's essentially impossible to attain perfection, varying both power AND grip is the inevitable outcome of our journey. Fixing one or the other is tempting, because it makes life easier, but it can only end in disappointment.

"1998 was a season when cars were undrivable: narrowed from 2 to 1.8m (less stability against roll), grooved tyres. I'm sure if I model 1997 results, grip will be higher. In 2000 again, only for 1999 I expect more grip problems."

Yes. This links back to something I mentioned before and also to a chat I had with TomMk a few years ago, when we were discussing how best to handle rule (and technology) changes - and one of the ideas Tom had was to create eras, to help simplify things, such as the ground effect era, the turbo era, the 3 litre era, the V10 era and the grooved tyre era etc. We never really pinned it down exactly but it is something I would like to think about again one day. Again your approach is more precise, but it tends to have serious implications for the track MDs as well as the PFs so it is an awful lot of work to do it right.

"But for me it's also important to feel the difference in drivability/engine power from one season to another."

Agreed. But again, the use of "eras" was an idea to make this slightly easier (and therefore quicker) to implement (because you'd have small groups of mods to work on, rather than having to do each one in turn). As an old guy, these considerations are increasingly important to me!

"Also I understand that underperformance on some circuits comes from car setup. My runs are all on the same setups which is non-sense, so you can achieve 1998 times just reducing wings in Monza/Hockenheim."

Oh yes, that's very important. I always make sure to feed back my preferred car setups to the CC cars as well (via the track MD, the magic data) and over the years I've done quite a bit of tweaking to the MD files because, being developed by different people at different times, they often lack consistency in the same way that the released mods often too. Again, that's not a criticism of anyone, because it takes YEARS to do all this to lots of tracks and there never was a uniform 'standard' to work to in any case, but well-refined MDs do bring a greater coherence to all mods - and given the great work you are doing with the PFs I'm sure you'll end up having a lot of fun with those too.

Here I've found what I wanted to find about variance:
[www.grandprixgames.org]

Suddenly, all the remarks and my own observations (about skill bonus of 60 per 100 variance) make a perfect sense.
25% of the variance is attributed before the start of the weekend, 75% in each session, in qualifying even for each run.

What is then the bonus due to variance?
Well, it depends. The mean value of uniformly distributed variable between 0 and 1 is equal to 0.5. So theoretically every 100 of variance must lead to +50 in driver skill.
But it is valid if you make only one qualifying attempt. Making several qualifying attempts means the best time is retained.
And the best of 2 uniformly distributed random variables between 0 and 1 is not equal to 0.5. It's 2/3.
The best of 3 such variables is =3/4. The best of 4 =4/5.
This means your driver skill increases with the number of qualifying attempts and strives to 1.

Though we should never forget about the weekend part of the variation. It's independent from the number of qualifying runs and its mean value = 0.5
So:
1 flying lap: bonus = 0.25*0.5+0.75*0.5=0.5
2 flying laps: bonus = 0.25*0.5+0.75*2/3=0.625
3 flying laps: bonus = 0.25*0.5+0.75*3/4=0.6875
4 flying laps: bonus = 0.25*0.5+0.75*4/5=0.725

On the other hand, it practically never reaches 0.725. I don't know if it's because I have few data (so the error is considerable, even 100 drivers attempts(or 5 qual x 22 drivers) will not give you enough data to decrease the error below 2%). But maybe there's another factor: fast cars can simply stuck behind slower cars, so when their random value is high, they cannot always convert their high driver skill into a good lap time.

One important consequence:
Your standard deviation drops with the number of runs.
Probability density function of a driver skill for 1 lap qualifying = 1. This means his max skill can be equally everywhere between 0 and 1.
For 2 laps it's equal to 2x (I omit here the weekend part). This means your driver is more likely to be at the right end of the variance.
For 3 laps it's 3x^2, for 4 laps it's 4x^3.

The mean will tend to 1, the field will be concentrated on the right side, but you'll still have some drivers on the left side with huge gaps to the leaders.
These lap times are so bad not because of mistakes, this is how the distribution is programmed. If you simulate 1 run with the same bhp, driver skills, variance*, you will definitely see high density at the top and low density at the bottom. And sometimes you have a higher standard deviation of lap times at lower variance values because the density on the left side is so low that you can have a bad driver at variance 500 and not have it at variance 1000. 1 run with 22 drivers is in this case insufficient to draw conclusions.

* Though variance must be high enough to distinguish it from internal variance which differs from circuit to circuit. And obviously don't forget the weekend part which is good for 25% of driver performance.

And: this counts for qualifying only, not for race. For race it's obviously 0.5.
Consequence #2: there's no need in higher variance for race. It's already de facto higher because several qualifying attempts artificially reduce variance. And if it's a 1-lap qualifying, its results are probably even random as race results.

An interesting study and a useful link to SDI's explanation of variance which I'd not seen before. Thanks.

I do have one question though. Towards the end of your last you state that "variance must be high enough to distinguish it from internal variance which differs from circuit to circuit". In what way does "internal variance" differ from circuit to circuit? Is there something comparable to variance in the magic data files?

Noog Schreef:
-------------------------------------------------------
> I do have one question though. Towards the end of
> your last you state that "variance must be high en
> ough to distinguish it from internal variance whic
> h differs from circuit to circuit". In what way do
> es "internal variance" differ from circuit to circ
> uit? Is there something comparable to variance in
> the magic data files?

As I can see from my simulation results with 0 variance (and this behaviour is reproduced on different bhp and grip combinations).
All tracks are from 1997 trackpack (I never verified on original 2001 circuits)
Imola: standard deviation = 0.065%
Barcelona: 0.084%
A1-Ring: 0.097%
Monaco: 0.102%
Montreal: 0.119%
Hungaroring: 0.135%
Nurburgring: 0.136%
Spa: 0.138%
Suzuka: 0.144%
Hockenheim: 0.153%
Magny-Cours: 0.160%
Silverstone: 0.160%
Monza: 0.160%
Melbourne: 0.164%
Buenos Aires: 0.200%
Interlagos: 0.210%

Again, this simulation is only performed once on each combination of bhp-grip, but for each bhp-grip combination you have similar results: Imola and Barcelona the closest distribution, Interlagos and Buenos-Aires the widest. The rest is somewhere in between.
I haven't analysed if it is a normal distribution. Almost all the results lie within 2 sigma or 95%. And it's a normal situation if you have 22 drivers. Statistically only one of them should be out of 2 sigma.

If it is related to magic files... I never touched them and neither I know what they are. When I touched the last time GP4, I only used GPxPatch, Easywad and manually imported carshapes and textures. And changed performances either via team editor either via GPxPatch.

Hmm. Again, very interesting. I edit track magic data files all the time so I don't have a untouched 1997 trackpack installation to look at right now (and I'm busy with other things), but there are one or two lines in the track MD files that, while not directly something labelled 'variance', could conceivably impact on the SD, because there are still some entries in the MD files that aren't totally understood.

However, it's also possible that the game EXE itself could be responsible for what you are measuring as track variance, because of the way it alters the wing settings of some of the computer controlled cars away from the values determined in the MD file. I don't recall exactly how it does this (although I do recall that it's not something we seem to be able to control), but there's an informative post about it somewhere on the forum and it does seem to be detectable in the on-track behaviour sometimes. Also, I've noticed over the years that a track set up in one slot doesn't always seem to behave in quite the same way when moved to another slot; which, if true, might suggest yet another set of parameters that we can't access. So, it's probably fortunate that you aren't a perfectionist!

Keep up the good work.

Again, I tested performance files. This time to estimate how good reliability was.

I found 32768 as 80% failure chance in GP3 editor. So basically my files are: 40960*failure percentage. And I got on average 46.6% of mechanical retirements although I expected 34.9%.
So this esteem is definitely too high. But: if I assume that 16384 is 100% failure, I will get lower failures on average than 34.9%.

Moreover, I made an interesting observation: Williams and Ferrari, teams with the lowest failures, were touched much less. So they had more points than expected. Ferrari won both drivers and constructors; Williams was by far 3rd: the team earned 2x more points than Benetton, or more than Benetton and Jordan combined.
On the other side, teams with high failure did not finish at all. There were some finishes, but failure rate for the least reliable teams is definitely overestimated.

This can mean that we're dealing with an exponential dependence (like radioactive decay), not linear.
I'll perform some quick tests with a uniform value for the whole peloton and will see if it's linear or not.

It does sound like you might be on to something, but for what it's worth that 16384 figure does have a long pedigree; dating back to at least 2008, if not before - and it's confirmed in this post by SDI a few years ago.

It would be nice to develop a task force to test the mods and produce more accurate performance to each one. We need a good driver ( not me rs ) a good engineer ( that knows performance files, track MD and engine physics ). I notice that some mods have a poor race start even if cc cars are fast enough in the race.

jcgrj Wrote:
-------------------------------------------------------
>I notice that some mods ha
> ve a poor race start even if cc cars are fast enou
> gh in the race.

It's a known problem bud. There are ways to fix it; but when you do it messes up the performance in the rest of the race.

There are quite a few posts about this problem on the forum, but ultimately we have little choice but to put up with it.

What some people do though, is they train themselves to hold their position until after the first corner.

It seems difficult at first, but once it becomes a habit it's not so bad!

Noog Schreef:
-------------------------------------------------------
> It does sound like you might be on to something, b
> ut for what it's worth that 16384 figure does have
> a long pedigree; dating back to at least 2008, if
> not before - and it's confirmed in this post by SD
> I a few years ago.


I tested today, and it's about 76.5% for 16384. In fact, it should be 80%, but some AI cars crash before they can have mechanical failures.
I remember also 100% was not achievable in early editors like GP3editor and others. This is why 16384 corresponds to 80%.
But it's surely linear: 10% for 2048, 20% for 4096, 40% for 8192. If 100% is theoretically achievable, I don't know, at this point I'm not interested in research beyond 16384.

But: I see that "didn't finish" is very rare (except Hockenheim where some run out of fuel), around 7-9%. So to come to the same retirement %, we have to correlate combine mech+didn't finish in GP4 with 1998 results. And, in other words, increase mechanical failures to achieve the same retirement %.

P.S. Do we have a topic where I can ask stupid questions? Like I want to modify nb of laps in the mod (I have 45 lap Hungaroring which is obviously mistake).

There is more than one way to adjust no of laps, but probably the most convenient is to use cmagic4_102.exe

Load cmagic4_102.exe and hit the browse button on the GP4 track file line and select your track dat.
Next, click the EditGP4Info button which has now come to life. That will pop up a little dialog.
If "laps" is listed there, change it. If not, type laps in the left hand column and the number of laps you want in the right, then hit save.

That's it!

Thanks for the info about failure rates. I'll be keeping an eye on mine.



Edited 2 time(s). Last edit at 01/22/2024 11:56PM by Noog.

I try to change some physics and MD to lower the problem insted of try to remain on the same position at race start.
It's very strange why this happens only in some mods. I think there's a better solution to solve that if we try to understand the cause ( maybe the track, physics, etc )

I experience this problem on MOD 80, 83 from Pedro ( maybe there are others )

Best

Jose


jcgrj Wrote:
-------------------------------------------------------
>I notice that some mods ha
> ve a poor race start even if cc cars are fast enou
> gh in the race.

It's a known problem bud. There are ways to fix it; but when you do it messes up the performance in the rest of the race.

There are quite a few posts about this problem on the forum, but ultimately we have little choice but to put up with it.

What some people do though, is they train themselves to hold their position until after the first corner.

It seems difficult at first, but once it becomes a habit it's not so bad!
Reply Quote Report

The causes are well understood - and there are ways to fix it for many tracks (power and physics); but not without messing up the performance in the rest of the race.

Setting the 'Rail line' to zero in the MD (Line 75 IIRC) can also help with some tracks, but not others.

Revisit the default 2001 game one day and you'll notice that the start line acceleration problem has always existed.

Thanks Noog!!!

Line 75 do you mean that one?

; desc75= Real-time factor, speeds up or slows down the in-game time (+/-1000 equals +/- 1.0)

maybe Im with the wrong description.

Jose

jcgrj Wrote:
-------------------------------------------------------
> Line 75 do you mean that one?

No, the line directly above that one. It's a number that tells the CC cars to hold their grid positions for a certain number of sectors. If you set it to zero, they should immediately try to look for ways to overtake, but as you'll see at Spa most obviously, they won't actually use all the track to do it (which is why you can nearly always sneak up on the right hand side before the first corner).

Here's how my MD files look in that section. Easier to understand probably.

desc73=no. of sectors at start that cars will hold grid position
desc74=real-world pole laptime (e.g. 70800 equals 1:10.800)
desc75=game pole laptime modifier (1000 = make CC cars 1s a lap faster, -3500 = 3.5s slower)

You want desc=73...

HTH.

Me again, here is an example of a performance file for Australia 1998

[Team #00]
Name=Williams,Mecachrome
Performance=765,775,2789
First Driver=1,Jacques Villeneuve,14465,263,15151,263
Second Driver=2,Heinz-Harald Frentzen,14704,263,14951,263

[Team #01]
Name=Ferrari,Ferrari
Performance=780,790,2300
First Driver=3,Michael Schumacher,15162,263,15073,263
Second Driver=4,Eddie Irvine,14644,263,14620,263

[Team #02]
Name=Benetton,Playlife
Performance=765,775,4973
First Driver=5,Giancarlo Fisichella,14601,263,14868,263
Second Driver=6,Alexander Wurz,14420,263,14430,336

[Team #03]
Name=McLaren,Mercedes
Performance=790,800,3693
First Driver=7,David Coulthard,15255,263,15286,263
Second Driver=8,Mika Hakkinen,15304,263,15304,263

[Team #04]
Name=Jordan,Honda
Performance=740,750,5538
First Driver=9,Damon Hill,14717,263,14763,263
Second Driver=10,Ralf Schumacher,14796,263,14766,263

[Team #05]
Name=Prost,Peugeot
Performance=755,765,15627
First Driver=11,Olivier Panis,14441,263,14040,263
Second Driver=12,Jarno Trulli,14537,263,14077,263

[Team #06]
Name=Sauber,Petronas
Performance=760,770,8888
First Driver=14,Jean Alesi,14371,263,14591,488
Second Driver=15,Johnny Herbert,14445,263,15008,263

[Team #07]
Name=Arrows,Arrows
Performance=690,700,16384
First Driver=16,Pedro-Paulo Diniz,14247,681,14184,263
Second Driver=17,Mika Salo,15153,263,14588,263

[Team #08]
Name=Stewart,Ford
Performance=725,735,9189
First Driver=18,Rubens Barrichello,14497,263,14462,263
Second Driver=19,Jan Magnussen,14217,263,14038,263

[Team #09]
Name=Tyrrell,Ford
Performance=700,710,16384
First Driver=20,Ricardo Rosset,13635,1687,14071,263
Second Driver=21,Toranosuke Takagi,14811,263,14767,263

[Team #10]
Name=Minardi,Ford
Performance=700,710,10018
First Driver=22,Shinji Nakano,14256,431,14128,263
Second Driver=23,Esteban Tuero,14580,263,14192,263

What are assumptions I made?
1. Reference value is 800bhp and 15406 grip (Hakkinen).
2. I used real engine powers from different sources (statsf1.com except Playlife and Honda which I found doubtful).
3. Variance of 263 corresponds to 0.6% in qualifying on average during the season. Yes, for different track it's different, but I prefer using the same variance across the season. 0.6% comes from GP4-qualifying distribution and my real calculations (based on 2020) of prediction error of driver result (SD=0.171%), so the delta max-min is about 0.58-0.59%, I rounded it up to 0.6% and it seems pretty realistic.
4. What if variance is different from 263? This means a driver has shown a very bad/good result on this exact circuit relative to his own performance throughout the season. In this case the variance is calculated differently:
4.1. His mean value was assumed to be his season mean +1SD for bad results.
4.2. Then the lower bound was assumed equal to his actual performance, the upper bound was calculated like if he had the same upper bound with variance of 263.
4.3. The difference between UB and LB is the new variance.
The same principle if a driver scored better (like Irvine in Monza). Here you can see only drivers that I adjusted up: Wurz, Alesi (qualifying), Diniz, Rosset, Nakano (race)
5. Generally speaking, I "restored" performance of early retired drivers (here Takagi, Diniz, Ralf Schumacher, Magnussen, Barrichello, Nakano) based on 2 things: their average race to qualifying gap (season) and field average race to qualifying gap (current track) in 50/50. The same principle was used for the entire field in rainy races like Silverstone and Spa.
6. For qualifying I used best weekend time instead of qualifying time. This changed A1-ring and partly Monza (Irvine was the best).

And it behaves pretty realistic in qualifying: poles mostly scored by Hakkinen/Coulthard/Schumacher. Eddie is among Williams and Benetton drivers. Alesi is somewhere in between Damon and Ralf. And Rosset totally sucks. No surprises like Barrichello in top-6 (unless it rains).

What I did not expect:
1. Race pace is quite different. There is obviously more impact of grip factor on lap times than in qualifying.
Irvine is assumed to perform better than Williams and Benetton, but he is outscored by both despite higher reliability. The reason is that Ferrari has a more powerful engine, so it is compensated by a lower grip. In qualifying he does slightly better than Williams-Benetton drivers, but in race it is not the case although the values I've given are superior to those of Mecachrome-powered drivers.

SO, BASICALLY THE CONCLUSION: AN APART RACE CONDITIONS CORRELATION IS NEEDED
(based on 1998 strategies of course, mostly 1 pit-stop, sometimes 2)

2. Grip has even more impact on a wet track (but this I knew long ago). Not critical for overall results as we have few rainy races and bad drivers are bad in rain conditions as well.

3. Bonus: if you start a non-championship race from the end of the grid, the starting grid is more chaotic than after a real qualifying. I've encountered Panis starting 3rd for example (although he was good in Brazil, he wasn't expected to be higher than 8th). My only explanations is that random generator makes only 1 simulation instead of 4 in real qualifying. Obviously you have more discrepancy (about 3 times).

I find it fascinating that even after 30 years we still know relatively little about seemingly arbitrary elements of the Sim such as how the performance files actually work. Geoff Crammond must be some kind of magician, no wonder its called magic data

I could be way off but there was a certain superstition that the infamous 16384 was the mystical element that everything worked around? It was treated like some kind of Cabalistic number that could not be violated. I could be totally wrong though, but I remember a lot of discussion around that number in the past. I'm pretty sure it was the Schumacher AI value in GP2

16384 is indeed a "round" number or 2^14 or 0x4000 in hexadecimal.
Anyway, there's no limit on this when you edit driver performance. I tried 17000 and it works perfectly.

Where is the limit, is car reliability. 16384 corresponds to failure rate of 80% and below this value the behaviour is linear. Above no clear dependency.

uncle_dominique Wrote:
-------------------------------------------------------
> 16384 is indeed a "round" number or 2^14 or 0x4000
> in hexadecimal.
> Anyway, there's no limit on this when you edit dri
> ver performance. I tried 17000 and it works perfec
> tly.
>
> Where is the limit, is car reliability. 16384 corr
> esponds to failure rate of 80% and below this valu
> e the behaviour is linear. Above no clear dependen
> cy.


Fascinating. According to Rremedio over on the gp2 forums who I consider to be authoritative and knowledgeable on these matters, the mythical 16384 also regulates the standard flow of time, fuel consumption, and semi pro grip (in gp2). How much if that is relevant to gp4 in your expertise?

He also said something about all the drivers performances being linked in some way to the fastest driver (Schumacher - 16384), but I can't find the post right now. It seems that the performances are operating in relationship to that, in gp2 at least