Sébastien Labbé

UPDATE (Oct 24, 2022), thanks to comments from reddit: fixed the way standard deviation is presented to avoid misinterpretation, removed the sin(-x) graphics.

When I started to play ultimate in September 2002 in Sherbrooke, the local team Stakatak was just coming back their very first (or maybe second?) participation at the Canadian Ultimate Championship, in the mixed division. I remember they lost all of their nine games and finished 16th out of 16 teams. But they came back in Sherbrooke with the "Spirit of the Game" award which we were very proud of.

What I want to discuss here is not whether the team that lose all its games and wins the Spirit of the Game deserves it or not. The question I want to consider in this blog post is about the evaluation of the spirit of the game in a typical ultimate frisbee game: are we biased by the end result of the game (win vs lose) when we evaluate the opponent's spirit of the game? In particular:

• do we give more spirit points to the opponent team when the opponent has lost against us?
• do we give less spirit points to the opponent team when the opponent has won against us?

The Spirit of the Game

As not everyone reading this post ever played an ultimate frisbee game, let me recall what is the spirit of the game and how it is evaluated nowadays in a tournament. As Ultimate (frisbee) is a self-officiated team sport, the spirit of the game is important. Every team thinks they have a good spirit but not every opponent agree. There are ways for teams to help (or force) them improve their spirit of the game, the most important one being the end of game discussion during which the two teams discuss the game and if necessary any issues that happenned during the game. Another is the evaluation of the spirit of the game by the opponent team, which is made by evaluating 5 subjects:

1. Rules Knowledge and Use (4 points)
2. Fouls and Body Contact (4 points)
3. Fair-Mindedness (4 points)
4. Positive Attitude and Self-Control (4 points)
5. Communication (4 points)

In Ultimate tournaments, the spirit scores of each team is public and allows to evaluate and rank each team. When a team is low ranked, it shows without ambiguity that the community thinks this team needs to improve because it was badly evaluated by more than one team. This peer-pressure contributes to make teams improve themselve. The ranking is also used to elect a most spirited team which is often given a "Spirit of the game" trophee at the end of the tournament.

Three tournaments considered for the data analysis

For the data analysis, we consider the following three tournaments that were held during Summer 2022:

• World Masters Ultimate Club Championships (WMUCC) 2022, Limerick, Ireland, June 25th to July 2nd 2022
• World Ultimate Club Championships (WUCC) 2022, Cincinnati Ohio, USA, July 23-30, 2022
• Canadian Ultimate Championships (CUC) 2022, Brampton, Ontario, August 18-21, 2022

These tournaments all use the Ultiorganizer website which allows to parse the results with the same Python script which I have made public.

In total, 1540 games were played in these 3 tournaments. Unfortunately, the score or the spirit score was not completed or is not available for all games. Maybe because teams forgot to provide the spirit scores or maybe the game was not played at all. We were able to access all needed data including final score and spirit scores for 1448 of the games (94 %). Since the spirit of both teams gets evaluated during a game, this means 1448 x 2 = 2896 evaluations of a team spirit.

Average Spirit Points

The average spirit points received by a team is shown in the table below for each of the three considered tournaments.

Spirit points are on average slightly above 10. Also, at WMUCC, the spirit points were higher in general, slightly above 11. We may interpret this as the fact that older master players playing for a long time were happy to play again at the international level after the pandemia and meet old friends which contributed to have nice spirited games in Limerick (why did not I try to go at Limerick again? I miss my old friends from Epoq or Nsom or Quarantine!).

Average Spirit Points for losers/winners

Now let's compare the average spirit points given to the loser of a game vs to the winner of a game. In the three considered tournaments, on average it turns out that the loser of the game always gets more spirit points. See the results in the following table.

We visualize below the distribution of spirit scores at WMUCC 2022 for losers vs winners with the following box plot graphics made with matplotlib through the pandas library. Small circle indicate what is called flier points. As mentionned in the matplotlib boxplot documentation, "flier points are those past the end of the whiskers".

Are losers more spirited in ultimate?

We can now answer the question asked in the title of this blog post and the answer is yes: the data says that losers are more spirited in Ultimate.

Or, alternatively, we can assume the hypothesis that losers and winners are equally spirited. This assumption implies that players must be biased by the end result of the game when evaluating the opponent's spirit of the game.

Lose-win Bias

It is natural to define the lose-win bias as the difference between the average spirit points obtained by the losing team and the winning team. In other words, how much more spirited are losers than winners? The results is in the following table.

At WUCC 2022 and CUC 2022, the losing team gets approximatively 0.5 more spirit points than the winning team. During WMUCC 2022, the losing team was obtaining 0.8 more spirit points than the winning team.

Interpretation

How can we interpret these results? Are losers really more spirited or can we accept that we are biased? Is there any other way to interpret the above results?

My interpretation is that we are biased by the end result which means winners and losers will say something like this (if I allow myself to caricature in a provocative way):

"Dear opponent, thanks for losing, we will give you one more spirit point for not making more effort."

"Dear opponent, thanks for the game, you won against us, but your communication was not so good, we give you one point less than we would have usually gave if you would have accepted to lose the game."

Of course I am volontarily exagerating and being a little provocative here to make us think about our own biases. We would never say sentences like this, but, basicaly, I think we might be actually doing this sometimes in a more disguised way.

I think that it is necessary that every ultimate frisbee player know about the existence of this lose-win bias in order to become more objective when evaluating the spirit of the game of the opponent.

Spirit score per score differential

I suggest now to go a bit further in the data analysis. Instead of splitting the spirit points according to the two win or lose cases, we can study the spirit points according to the score differential. This should allow us to answer interesting questions such as:

• Is winning by 1 point the worse thing to do to get a good spirit?
• By how many points should a team win to expect the most spirit points?
• By how many points should a team win to leverage the spirit bias?

Below is a graphics which shows the average spirit point obtained by a team according to the point differential during the three considered tournaments:

We observe that spirit scores were in general higher at WMUCC 2022. Also, we can see that each curve reach its minimum around +1 or +2, which means you want to win by more than one or two points if you want to win and maximize your spirit points.

Bias per score differential

In what follows, we will discuss the bias per score differential. Here is the graphics summarizing the average lose-win bias according to the end of game point differential (one broken line per tournament):

Let's first try to explain the above graphics. The x-axis shows by how much you won the game: +2 means you won the game by 2, and -5 means you lost the game by 5. The graphics shows for each score difference, the average difference between your team spirit score and the opponent spirit score for each of the three tournaments. Take for example the case when your team win by 2, the graphics shows that on average in all of the three tournaments, the spirit score you get is 1 less than the losing team.

Some similarities appear in the three tournaments. On the right part of the graphics, for positive values on the x-axis, the graphic lines are below zero whereas for negative values on the x-axis, the graphic lines are above zero. This essentially means that losers have on average a better spirit evaluation than winners.

One could have expected that winning by one point is worse than winning by two points, because it is in this kind of game that a single action where a travel or fault is called that may affect the outcome of the game, thus affecting the spirit results. But the data shows that winning by 2 is worse than winning by 1 in terms of spirit bias. A possible interpretation goes as follows. When you lose on the universe point, you show to everyone that you were very close to win which is a honorific way of losing. On the other hand, losing by 2 does not allow you to pretend you were close enough to win the game. This may explain why the spirit bias is higher for game finishing by a difference of 2 points compared to 1 point.

Another pattern which is common in each of the three lines is that a local minimum for the lose-win bias is reached when winning by 2. It seems that winning by a higher margin (3, 4, 5 or 6 points) makes the lose-win bias globally closer to zero.

My personnal interpretation is as follows. Winning by 1 or 2 points is not good for your spirit points, because you basically allow your opponent to think that they could have won the game (which may make them biased when evaluating your spirit points). Winning by 5 or 6 points seems to neutralize the win-lose bias. This is enough a point difference which establishes a hierarchy and makes the opponent accept their lost, but not too much that part of the game become meaningless which may impact the fun of both team to play the game.

When the score difference increases, what happens is more chaotic, so I don't know if we can make any safe interpretations, but it seems winning by 7, 8 or 9 is not good for your spirit. And then, winning by exactly 10 points seems also to neutralize the bias. There are fewer games finishing by a difference of more than 10 points, so I will not discuss their statistics here.

Conclusion

To conclude, I would like to recall the real objective of this blog post which is to make the ultimate frisbee players acknowledge the existence of biases when evaluating the opponent's spirit of game, one bias being related to the score result outcome. Once this is acknowledged, the next step is to search for ways to leverage the bias. This task belongs to each and every ultimate frisbee players.

Code and data

I made my code public allows to reproduce the computations and graphics. Everything is in this gitlab repository:

https://gitlab.com/seblabbe/spirit-bias-in-ultimate

The code is written in Python. Data is downloaded with urllib library and stored as csv files. Parsing of Ultiorganizer websites is done in a Python script ultiorganizer_parser.py that I wrote. Analysis of the csv files is done with pandas library with Jupyter notebooks. Graphics are made with matplotlib.

Since Sage Days 20 at Marseille held in January 2010, I have been doing the same example over and over again each time I showed someone else how object oriented coding works in Python: using fruits, strawberry, oranges and bananas.

Here is my file: fruit.py. I use to build it from scratch by adding one line at a time using attach command to see what has changed starting with Banana class, then Strawberry class then Fruit class which gathers all common methods.

This time, I wrote the complete documentation (all tests pass, coverage is 100%) and followed Sage coding convention as far as I know them. Thus, I hope this file can be useful as an example to explain those coding convention to newcomers.

One may check that all tests pass using:

$ sage -t fruit.py
sage -t  "fruit.py"
[3.7 s]

----------------------------------------------------------------------
All tests passed!
Total time for all tests: 3.8 seconds

One may check that documentation and doctest coverage is 100%:

$ sage -coverage fruit.py
----------------------------------------------------------------------
fruit.py
SCORE fruit.py: 100% (10 of 10)
----------------------------------------------------------------------