Research Protocol

By Dr. Robert Lodder , Ms. Cynthia Dickerson
Corresponding Author Dr. Robert Lodder
Pharmaceutical Sciences, BPC223 Biopharmaceutical Complex - United States of America 40536
Submitting Author Dr. Robert A Lodder
Other Authors Ms. Cynthia Dickerson
Pharmaceutical Sciences, University of Kentucky - United States of America


benchmark dose, toxicity, bootstrap, BEST, extended BEST

Lodder R, Dickerson C. A Novel Statistical Approach to NOAEL: QBEST Applied to Dosing of Ellagic Acid. WebmedCentral TOXICOLOGY 2018;9(6):WMC005451

This is an open-access article distributed under the terms of the Creative Commons Attribution License(CC-BY), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Submitted on: 02 Jun 2018 02:59:02 AM GMT
Published on: 05 Jun 2018 05:26:36 AM GMT


QBEST, a novel statistical method, can be applied to the problem of estimating the No Observed Adverse Effect Level (NOAEL) of a New Molecular Entity (NME) in order to anticipate a safe starting dose for beginning clinical trials. The NOAEL from QBEST (called the QNOAEL) can be calculated using multiple disparate studies in the literature and/or from the lab. The QNOAEL is similar in some ways to the Benchmark dose (BMD) and is superior to the BMD in others. The Benchmark Dose Method is currently widely used in toxicological research.

Results are used in a simulation based on nonparametric cluster analysis methods to calculate confidence levels on the difference between the Effect and the No Effect studies. The QNOAEL simulation generates an intuitive curve that is comparable to the dose-response curve. The NOAEL of ellagic acid (EA) will be calculated for clinical trials of its use as a component therapeutic agent (in BSN476) for treating Chikungunya infections. This will be the first published application of QBEST to the problem of NOAEL determination. The specific aims of the proposed study are to evaluate the accuracy and precision of the QBEST Simulation and QNOAEL compared to the Benchmark Dose Method, and to calculate the QNOAEL of EA for BSN476 Drug Development.

Specific Aims

Nonparametric statistics are statistics that are not based on parameterized families of probability distributions, like the normal distribution. They are important because is it common for data to follow a distribution other than a known one, like the normal distribution.

The NOAEL is an important part of the non-clinical risk assessment for new drugs like BSN476, a drug for treating Chikungunya. The NOAEL is a professional opinion based on the design of the study, indication of the drug, expected pharmacology, and spectrum of off-target effects. It is the highest dose at which there was not an observed toxic or adverse effect 1. There are important theoretical limitations to the traditional NOAEL calculation, which led to the newer Benchmark Dose method, which also has a number of problems. In brief, the traditional NOAEL is determined by administering a few different doses of drug to a group of subjects, observing those subjects for physiological change, and assigning the dosages to the categories of "having an adverse effect" and "not having an adverse effect". The highest dosage resulting in no adverse effect is determined to be the NOAEL.

The NOAEL method is problematic because (1) dose levels are often an order of magnitude apart, and it is highly unlikely that the exact NOAEL dosage will be administered in any particular study; (2) determination of what constitutes an "effect" can be difficult when negative effects are of a highly subjective nature (for example, when mood is affected)2. A nonparametric simulation using QBEST (Quantile Bootstrap Error-adjusted Statistical Test) can solve the problems associated with the use of the traditional NOAEL or the Benchmark Dose (BMD) and enable accurate toxicity estimates to be made.

Specific Aim 1. Evaluate the accuracy and precision of the QBEST Simulation and QNOAEL compared to the Benchmark Dose Method.
Utilizing synthetic data with known characteristics, the BMD and QNOAEL will be calculated. The QNOAEL will then be compared to the BMD to determine which is closest to be known answer for the synthetic data.

Specific Aim 2. Calculate the QNOAEL of Ellagic Acid for BSN476 Drug Development.
Chikungunya is a rapidly spreading mosquito-borne disease that now infects over 3 million people worldwide 3. BSN476, a drug for treating chikungunya infections, contains in part EA. QBEST will estimate a safe level of EA for the first-in-human study in order to develop a treatment for Chikungunya. An EA toxicity meta-analysis using food consumption will be completed as part of the Investigational New Drug (IND) application to the FDA. Studies will be selected from the literature and analyzed according to the Cochrane protocols, and the QNOAEL of EA will be calculated along with the NOAEL and BMD. These results will serve as the basis for the first-in-human study of EA.

Research Strategy

Method Significance.  The NOAEL depends strongly on the dose selection, dose spacing, and sample size of a single study from which the critical effect has been identified.  The primary goal of BMD modeling is to define a point of departure that is largely independent of study design. But while the BMD effectively enables multiple studies to be pooled to increase accuracy, it does not handle studies with conflicting results gracefully, as will be seen below 4.

The new QBEST nonparametric meta-analysis of multiple studies so far appears to be superior to BMD modeling (the current toxicological "gold standard"). Unlike BMD, the QNOAEL estimate is not limited by the format of the data presented. The QNOAEL is no more time-consuming to calculate than the BMD, and provides a simpler decision-making process. For example, the graphs below show the BMD and QBEST simulations for THC in hemp seed. Note that the same clinical studies were used for both analyses. QBEST not only more clearly demonstrates the trend of data, but also produces a correlation curve which is intuitively noncontradictory. (It is to be expected that different studies may reach contradictory results on the effects of a given dosage, as study methods and populations vary.)

Application Significance. Chikungunya is a rapidly spreading mosquito-borne disease that now infects over 3 million people worldwide 3. The disease originated in Africa around 1700 A.D., and until recent years, reported infections were limited to the African continent and Southeast Asia 5. The disease was first identified in 1952 during an outbreak so serious that infections were clinically indistinguishable from dengue fever 6. Throughout the 1960s and 1970s, outbreaks were reported in Southeast Asia6. After decades without another Southeast Asian outbreak, a 1999 outbreak in Indonesia led to a massive outbreak reported in India in 2006, the strain responsible for this resurgence bearing 99% similarity to the strain responsible for a 1989 outbreak in Uganda 4,6. In December, 2013, the disease made its debut in the Americas, with its first local transmission occurring on the island of St. Martin; local transmission in French Guiana on the South American continent occurred later that month 7-9. After only two years, local transmission had been documented in 19 Caribbean countries, including Puerto Rico, as well as in nearly every country on the South American continent 7-10. The WHO has currently issued a level-1 watch for travelers visiting South America and the Caribbean and expects Chikungunya to spread 7,8.

BSN476 contains in part EA. EA is a polyphenolic compound with antiproliferative and antiviral properties11. EA at 10 uM produces 99.6% inhibition of Chikungunya virus in vitro 11. EA is found in a number of plant extracts, usually in the form of hydrolyzable ellagitannins which are complex esters of EA with glucose 12,13. Ellagitannins are broken down in the intestine to eventually release EA 12,13. To develop BSN476 as a treatment for Chikungunya, the PK of the drug must be studied in a first-in-human (FIH) trial, and a safe range of exposure must be determined for that trial.

QBEST is applied within the Cochrane framework for meta-analysis (the Cochrane framework provides a "Garbage-In-Garbage-Out" standard for data inputs - generally clinical studies) to determine doses for the first in human study 14. The QNOAEL of EA will be estimated from previously published food consumption studies. Bootstrap replication and manipulation of data clusters will reveal the QNOAEL of EA with 98% confidence.
This project will use the QNOAEL to estimate a safe range of exposure to EA for the FIH study on the way to developing a treatment for Chikungunya. QBEST is a robust O(n1) algorithm that is designed for massively parallel computers, and is a very powerful meta-analysis tool (most algorithms use matrix factorization and are O(n3) in execution time). The QBEST algorithm will be translated from MATLAB into Python to make it more accessible to the scientific community. Very large sample datasets will be used to test the memory usage and reproducibility of the results of the algorithm, as well as minimum parameters for its usage. These data have permitted estimation of the maximum analysis capability of cloud computing services and the National Science Foundation XSEDE supercomputer (Comet).


This proposal utilizes a statistical method designed for Big Data problems, QBEST, inside a new simulation to estimate the QNOAEL for a drug with 98% confidence from a set of small studies 15,15-18. QBEST is a form of cluster analysis, which is a common analytical technique for determining chemical identity and purity 15-18. So far, the QNOAEL appears to be superior to the NOAEL and the BMD.

QBEST is applied within the Cochrane framework for meta-analysis.  QBEST works by analyzing clusters of studies that found an effect, and clusters of studies that found no effect (the studies can use different dose levels).  By analyzing the quantiles of each cluster and adjusting for cluster skew, QBEST can measure the distance between clusters in probability space, until it finds the dose that yields no adverse effects for the entire human population at a specified level of statistical significance (see Fig. 6, 98% level set by default).


This is a software development project undertaken as part of a larger drug development project. Good Engineering Practice, Standard Operating Procedures (SOPs) and working practice guidelines have been implemented for project design as well as execution (see Fig. 7). A robust change control system must be implemented in this project.

The Design SOPs and Configuration Management system will be applied to the system designed to resolve each specific aim in this project. Each specific aim will begin with a Needs Analysis to determine what the new system needs to be able to do. A requirements analysis will also be conducted to determine what is required to fill those needs. A System Requirements Review (SRR) will demonstrate understanding of the requirements documents (scope, specifications, schedule, validation plans, and budget). SRR will determine the initial design direction and describe preliminary data and progress, and how these will converge to an optimum and complete system configuration for the specific aim.The memory needed to run QBEST on large datasets and evaluate the performance of the algorithm must be quantified. (Very large sample datasets to test the memory usage and reproducibility of the results of the QBEST algorithm are being created, as well as to set ranges of parameters for its usage. Parameters include the number of bootstrap replications desired (b), the number of variables in the multivariate analysis (d), and the size of the dataset used (n). Preliminary data indicate that the typical laptop computer can process over one million bootstrap replication samples, independent variables, or sample data points when the other two parameters are minimized, or over one-thousand bootstrap replications, variables, and data points [all maximized at approximately 1000 inputs]).

As the system evolves through the development process, topic experts will be invited to later design reviews (especially CDR, TRR, and MRR). System Design Review (SDR) acts as a control gate that reviews and approves the top-level system design solution and rationale 19. It is the decision point to proceed with system specification flow down to individual physical and process configuration items 19. System limitations will be refined at SDR.

Using the run-time and memory usage data determined in Specific Aim 1, the performance capabilities of the algorithm on an NSF supercomputer (XSEDE Comet) will be calculated. QBEST is capable of tackling immensely large datasets, and the computing capabilities of the algorithm will be stretched on a massively parallel machine to demonstrate proof-of-concept. An SDR report will be added to the Design History file for FDA.

A Preliminary Design Review (PDR) will be performed on each configuration item or group of configuration items to: (1) Evaluate the progress, technical acceptability, and risk resolution, (2) Measure its harmony with performance and engineering specialty requirements of the Configuration Item development specification, (3) Evaluate the extent of definition and evaluate the technical risk connected with the selected methods/processes, and (4) Demonstrate the existence and compatibility of the physical and functional interfaces among the configuration item and other items of equipment, facilities, computer software, and personnel 19. Topic experts will also be invited to the review. A Blue team and a Red team are used for design and validation, respectively. A PDR report will be added to the Design History file for FDA in the annual reporting system.

Critical Design Review (CDR) is the last design review conducted before an action is taken that is irreversible. (1) CDR is a review to establish that detail design of the configuration item under review meets cost, schedule, and performance requirements. (2) CDR will establish detail design compatibility among the configuration item and other items of equipment, facilities, computer software and personnel. (3) CDR will gauge configuration item risk areas (on a technical performance, cost, and schedule basis). Topic experts will again be invited to the review. A Blue team and a Red team are used for design and validation, respectively. A CDR report will be added to the Design History file for FDA in the annual reporting system.

Deployment Readiness Review (DRR) is held to confirm readiness for deployment. This review is conducted to ensure that all deficiencies are corrected before actual use. The complete system is challenged every feasible way (conceptually, physically, cyber-, etc.). The DRR demands the review and analysis of all subsystem/unit level testing preceding the formal acceptance tests. Topic experts will be invited to the review. A Blue team and a Red team are used for design and validation, respectively. A DRR report will be added to the Design History file for FDA.

The QBEST algorithm will be translated into Python to make it more accessible to the scientific community. Once QBEST is available in Python it will run on Amazon Web Services, Microsoft Azure, and Google Compute Engine as well as the NSF XSEDE Comet supercomputer currently being used. The REPLICA algorithm has already been translated from Matlab into Python to make it more accessible to the scientific community. However, the algorithm on which QBEST relies has yet to be translated and the entirety of the program remains to be validated.


1. NIH U.S.National Library of Medicine: Tox Tutor . NOAEL and LOAEL. 2017. 7-17-2017.
2. Esdaile, D. J. Principles, benefits and limitations of the NOAEL approach. Toxicology Letters 1994, 74 (Supplement 1), 23.
3. Seppa, N.; Hirshfeld, J. Chikungunya is on the move. Science News . 6-25-2015. The Society for Science & the Public. 7-27-2017.
4. Davis, J. A.; Gift, J. S.; Zhao, Q. J. Introduction to benchmark dose methods and U.S. EPA's benchmark dose software (BMDS) version 2.1.1. Toxicology and Applied Pharmacology 2011, (254(2)), 181-191.
5. Cherian, S. S.; Walimbe, A. M.; Jadhav, S. M.; Gandhe, S. S.; Hundekar, S. L.; Mishra, A. C.; Arankalle, V. A. Evolutionary rates and timescale comparison of Chikungunya viruses inferred from the whole genome/E1 gene with special reference to the 2005 outbreak in the Indian subcontinent. 2009, 9 (1), 16-23.
6. World Health Organization Regional Office for South-East Asia . Surveilance and Outbreak Alert: Chikungunya. 2017. 7-27-2017.
7. US Center for Disease Control and Prevention . Travel Notice: Chikungunya in the Caribbean. 9-17-2015. 7-27-2017.
8. US Center for Disease Control and Prevention . Chikungunya in South America. 9-7-2016. 7-27-2017.
9. US Center for Disease Control and Prevention . Chikungunya Virus: Geographic Distribution. 4-22-2016. 7-27-2017.
10. US Center for Disease Control and Prevention . Chikungunya Nowcast for the Americas. 7-27-2017. 7-27-2017.
11. Kaur, P. e. a. Inhibition of Chikungunya Virus Replication by Harringtonine, a Novel Antiviral that Suppresses Viral Protein Expression. Antimicrobial Agents and Chemotherapy 2013, 57 (1), 155-167.
12. Tomás-Barberán, F. A.; García-Villalba, R.; González-Sarrías, A.; Selma, M. V.; Espín, J.; . Ellagic Acid Metabolism by Human Gut Microbiota: Consistent Observation of Three Urolithin Phenotypes in Intervention Trials, Independent of Food Source, Age, and Health Status. J. Agric. Food Chem. 2014, 62 (28), 6535-6538.
13. Espín, J.; Larrosa, M.; García-Conesa; Tomás-Barberán, F. A. Biological Significance of Urolithins, the Gut Microbial Ellagic Acid-Derived Metabolites: The Evidence So Far. Evidence-Based Complementary and Alternative Medicine 2013, 2013.
14. Cochrane Handbook for Systematic Reviews of Interventions. Higgins JPT and Green, S. [Version 5.1.0]. 2011. The Cochrane Collaboration.
15. Lodder, R. A. Method for detecting subpopulations in spectral analysis. 6-9-1992. Google Patents.
16. Lodder, R. A.; Hieftjet, G. M. Detection of Capsule Tampering by Near-Infrared Reflectance
Analysis. Analytical Chemistry 1987, 59 (15), 1921-1930.
17. Lodder, R. A.; Hieftje, G. M. Detection of Subpopulations in Near-Infrared Reflectance Analysis. Applied Spectroscopy 1988, 42 (8), 1500-1512.
18. Lodder, R. A.; Hieftje, G. M. Quantile BEAST Attacks the False-Sample Problem in Near Infrared Reflectance Analysis. Applied Spectroscopy 1988, 42 (8), 1353-1365.
19. Shamieh, C. The V-model for the systems engineering process. In Systems Engineering for Dummies, Wiley Publishing, Inc.: Hoboken, NJ, 2011.

Source(s) of Funding

The project described was supported in part by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1TR001998. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.  This project was also supported by NSF ACI-1053575 allocation number BIO170011.

A complete version with figures is available at

Competing Interests


0 reviews posted so far

0 comments posted so far

Please use this functionality to flag objectionable, inappropriate, inaccurate, and offensive content to WebmedCentral Team and the authors.


Author Comments
0 comments posted so far


What is article Popularity?

Article popularity is calculated by considering the scores: age of the article
Popularity = (P - 1) / (T + 2)^1.5
P : points is the sum of individual scores, which includes article Views, Downloads, Reviews, Comments and their weightage

Scores   Weightage
Views Points X 1
Download Points X 2
Comment Points X 5
Review Points X 10
Points= sum(Views Points + Download Points + Comment Points + Review Points)
T : time since submission in hours.
P is subtracted by 1 to negate submitter's vote.
Age factor is (time since submission in hours plus two) to the power of 1.5.factor.

How Article Quality Works?

For each article Authors/Readers, Reviewers and WMC Editors can review/rate the articles. These ratings are used to determine Feedback Scores.

In most cases, article receive ratings in the range of 0 to 10. We calculate average of all the ratings and consider it as article quality.

Quality=Average(Authors/Readers Ratings + Reviewers Ratings + WMC Editor Ratings)