promote the need for models that can be used for bi- or multi-modal distributions. While this is interesting, the authors have not made a convincing case that this is a pressing need. It would have been useful if they could supply some information on how often such data arise in a regulatory submission.
The authors address the problem of bi-modality which is sometimes observed in data used for SSDs. They introduce this by noting that “the toxicity of plants and animals to an herbicide is, by design, going to be markedly different. In such cases, the empirical SSD will exhibit bimodality.” This is unfortunate, since for regulatory risk assessment, there is already a recognition that different SSDs are required for different segments of species at risk, e.g., aquatic plants, terrestrial plants, and aquatic invertebrates are not mixed together. So to some extent, this seems to be a solution in search of a problem. The use of mixture models to address this topic is certainly interesting in those situations where it occurs. Of greater interest is the prevalence of datasets for regulatory risk assessment that have this problem. No information is provided on that. Having done many SSDs for such use, I have rarely encountered bi- or multi-modality frequently and wonder whether this arises mostly from an attempt to increase the number of species values to use by mixing incompatible groupings. The authors acknowledge that mixture models require estimating more model parameters, and this narrows the applicability and practicality of this idea.
RESOLUTION
Beginning of section on Multimodality now reads:
In our experience, multimodality of the empirical SSD is not uncommon. This arises because the toxicity data underpinning the empirical SSD are not from a single, common probability model as is conventionally assumed. The use of toxicity data that relate to different taxonomic groups, endpoints, test durations, modes of action, or sensitivities will often result in multimodal SSDs. At the very least, a somewhat arbitrary dichotomy is usually identified based on test organisms being ‘more’ or ‘less’ sensitive to the toxicant under consideration.
As an obvious example of bimodality, the toxicity of plants and animals to an herbicide is, by design, going to be markedly different. In such cases, the empirical SSD will exhibit bimodality (that is, the distribution has two modal values). We acknowledge that, in a regulatory context, the usual advice is to fit different SSDs to different segments of species at risk (e.g. aquatic plants, terrestrial plants, and aquatic invertebrates). However, as previously noted, this may be impractical if insufficient data are available in any or all subgroups to meaningfully fit an SSD. Further, as we do not know the exact mode of action of many substances, bi-modality cannot be ruled out.
ECCC has not tested for bi-modality for anything other than pesticides so we don’t have an example we could contribute. However, I agree that there is no reason to assume uni-modality so why rule out bi-modality?, especially when you are working with substances with unknown mode of actions and not enough taxa-specific data for separate SSDs.
To take a step back, from a first principles perspective, we do not have an underlying theory that suggests that species sensitivities follow a single uni-modal distribution. When the mode of action of the substance targets a specific sub-group of species (i.e. pesticides or herbicides), then clearly we see bi-modality. But as we do not know the exact mode of action of many substances, bi-modality cannot be ruled out. Therefore, there is no reason to assume uni-modality and methods for testing for bi-modality are helpful.
Note that the ammonia data set provided by Rick a while back (see #18) fits the mixture very well, and only has a small number of plant species. Adding that example could be useful for addressing more than one point.
Re the larger comment, I emailed Rick:
I am trying to think of examples where we found bimodality apart from SSDs for herbicides. PFOS was on the borderline but ssdtools didn’t show that. For the tropical Ni marine data bimodality was possible but the evidence was dismissed. Can you think of any others to make our case stronger? Dioxin is one where all of the sensitive species are fish, but we didn’t check modality in 2017 when that was done by John Chapman. Maybe it is a solution in search of a problem taken over by the statisticians.
Hopefully Rick will save us.
I don’t think it’s a solution looking for a problem. In my experience at looking at SSDs, bi (and to a lesser degree, multi) modality is fairly common. It will be highly pronounced when there are obvious groupings/modes of action in the combined data set, but I think it’s also a data artifact. I’ve always been of the view that there is an underlying probability distribution for each species’ toxicity value (and if you’re a fan of TK-TD models, then you’d have to believe there was an underlying probability model for each organism). So when you aggregate species tox. data, distinct groupings may be evident. This gives rise to multi-modality. Whatever the cause/reason, it is an observed phenomenon that must be dealt with statistically otherwise you’re HCx values are rendered pretty useless.
As emailed to Graeme this morning:
Bimodality has been an issue mostly for pesticides. However, we could include dioxins in the list because John did test for, and found, significant differences between fish and other taxa (thus, the dataset was multi/bi-modal). Michael or Reinier would be able to provide some stats as to how many of the pesticide GVs that they’ve been deriving have exhibited bimodality. The reviewer made a valid point in that it is desirable to construct SSDs for different taxonomic groups. This was also pointed out by Michael Newman in his 2000 paper. However, often there can be insufficient data to derive robust taxa-specific SSDs (e.g. sulfometuron-methyl). In such cases, the use of multi-modal mixture models is of use. Also, the reviewer refers to the use of SSDs in risk assessments, which can often be more targeted and detailed than generic (default) WQBs for ‘whole of aquatic ecosystem’ protection. Mixture models may play more of a role for the latter than the former. We could acknowledge all of the above in a couple of sentences in the ms.
Again there are two item 16s listed so only one can be commented on. Re the issue of bimodality, I would comment to the reviewer:
Tests for bimodality are required in Australia as part of the derivation of default guideline values. In many cases, a poor SSD fit is improved if tests reveal bimodality and a more reliable GV is derived.
RE: Duplicated item numbers – maybe a bit confusing, but same number used for 2 or more comments about the same thing.