Ultrasound-mediated delivery of doxorubicin to the brain results in immune modulation and improved responses to PD-1 blockade in gliomas.
Doxorubicin
/ pharmacology
Animals
Humans
Programmed Cell Death 1 Receptor
/ antagonists & inhibitors
Mice
Blood-Brain Barrier
/ metabolism
Brain Neoplasms
/ drug therapy
Microbubbles
Cell Line, Tumor
Glioma
/ drug therapy
Brain
/ metabolism
Female
Drug Delivery Systems
Ultrasonic Waves
Glioblastoma
/ drug therapy
Male
Microglia
/ drug effects
Mice, Inbred C57BL
Antibodies, Monoclonal, Humanized
/ therapeutic use
Immune Checkpoint Inhibitors
/ pharmacology
Polyethylene Glycols
Journal
Nature communications
ISSN: 2041-1723
Titre abrégé: Nat Commun
Pays: England
ID NLM: 101528555
Informations de publication
Date de publication:
06 Jun 2024
06 Jun 2024
Historique:
received:
08
03
2023
accepted:
29
04
2024
medline:
7
6
2024
pubmed:
7
6
2024
entrez:
6
6
2024
Statut:
epublish
Résumé
Given the marginal penetration of most drugs across the blood-brain barrier, the efficacy of various agents remains limited for glioblastoma (GBM). Here we employ low-intensity pulsed ultrasound (LIPU) and intravenously administered microbubbles (MB) to open the blood-brain barrier and increase the concentration of liposomal doxorubicin and PD-1 blocking antibodies (aPD-1). We report results on a cohort of 4 GBM patients and preclinical models treated with this approach. LIPU/MB increases the concentration of doxorubicin by 2-fold and 3.9-fold in the human and murine brains two days after sonication, respectively. Similarly, LIPU/MB-mediated blood-brain barrier disruption leads to a 6-fold and a 2-fold increase in aPD-1 concentrations in murine brains and peritumoral brain regions from GBM patients treated with pembrolizumab, respectively. Doxorubicin and aPD-1 delivered with LIPU/MB upregulate major histocompatibility complex (MHC) class I and II in tumor cells. Increased brain concentrations of doxorubicin achieved by LIPU/MB elicit IFN-γ and MHC class I expression in microglia and macrophages. Doxorubicin and aPD-1 delivered with LIPU/MB results in the long-term survival of most glioma-bearing mice, which rely on myeloid cells and lymphocytes for their efficacy. Overall, this translational study supports the utility of LIPU/MB to potentiate the antitumoral activities of doxorubicin and aPD-1 for GBM.
Identifiants
pubmed: 38844770
doi: 10.1038/s41467-024-48326-w
pii: 10.1038/s41467-024-48326-w
doi:
Substances chimiques
Doxorubicin
80168379AG
Programmed Cell Death 1 Receptor
0
liposomal doxorubicin
0
pembrolizumab
DPT0O3T46P
Antibodies, Monoclonal, Humanized
0
Immune Checkpoint Inhibitors
0
PDCD1 protein, human
0
Polyethylene Glycols
3WJQ0SDW1A
Types de publication
Journal Article
Langues
eng
Sous-ensembles de citation
IM
Pagination
4698Subventions
Organisme : U.S. Department of Health & Human Services | National Institutes of Health (NIH)
ID : 1R01NS110703-01A1
Organisme : U.S. Department of Health & Human Services | National Institutes of Health (NIH)
ID : 1U19CA264338-01
Organisme : U.S. Department of Health & Human Services | National Institutes of Health (NIH)
ID : P50CA221747
Organisme : U.S. Department of Health & Human Services | NIH | National Cancer Institute (NCI)
ID : 1R01CA245969-01A1
Informations de copyright
© 2024. The Author(s).
Références
Banks, W. A. From blood-brain barrier to blood-brain interface: new opportunities for CNS drug delivery. Nat. Rev. Drug Discov. 15, 275–292 (2016).
pubmed: 26794270
doi: 10.1038/nrd.2015.21
Sarkaria, J. N. et al. Is the blood-brain barrier really disrupted in all glioblastomas? A critical assessment of existing clinical data. Neuro Oncol. 20, 184–191 (2018).
pubmed: 29016900
doi: 10.1093/neuonc/nox175
Eidel, O. et al. Tumor infiltration in enhancing and non-enhancing parts of glioblastoma: a correlation with histopathology. PLoS One 12, e0169292 (2017).
pubmed: 28103256
pmcid: 5245878
doi: 10.1371/journal.pone.0169292
Brandes, A. A. et al. Recurrence pattern after temozolomide concomitant with and adjuvant to radiotherapy in newly diagnosed patients with glioblastoma: correlation With MGMT promoter methylation status. J. Clin. Oncol. 27, 1275–1279 (2009).
pubmed: 19188675
doi: 10.1200/JCO.2008.19.4969
Gebhardt, B. J. et al. Patterns of failure for glioblastoma multiforme following limited-margin radiation and concurrent temozolomide. Radiat. Oncol. 9, 130 (2014).
pubmed: 24906388
pmcid: 4055938
doi: 10.1186/1748-717X-9-130
Zhang, D. Y. et al. Ultrasound-mediated delivery of paclitaxel for glioma: a comparative study of distribution, toxicity, and efficacy of albumin-bound versus cremophor formulations. Clin. Cancer Res. 26, 477–486 (2020).
pubmed: 31831565
doi: 10.1158/1078-0432.CCR-19-2182
Sabbagh A., et al. Opening of the blood-brain barrier using low-intensity pulsed ultrasound enhances responses to immunotherapy in preclinical glioma models. Clin Cancer Res. May; https://doi.org/10.1158/1078-0432.CCR-20-3760 , (2021).
Hynynen, K., McDannold, N., Sheikov, N. A., Jolesz, F. A. & Vykhodtseva, N. Local and reversible blood-brain barrier disruption by noninvasive focused ultrasound at frequencies suitable for trans-skull sonications. Neuroimage 24, 12–20 (2005).
pubmed: 15588592
doi: 10.1016/j.neuroimage.2004.06.046
McDannold, N., Vykhodtseva, N., Raymond, S., Jolesz, F. A. & Hynynen, K. MRI-guided targeted blood-brain barrier disruption with focused ultrasound: histological findings in rabbits. Ultrasound Med Biol. 31, 1527–1537 (2005).
pubmed: 16286030
doi: 10.1016/j.ultrasmedbio.2005.07.010
Meng, Y. et al. MR-guided focused ultrasound enhances delivery of trastuzumab to Her2-positive brain metastases. Sci. Transl. Med. 13, eabj4011 (2021).
pubmed: 34644145
doi: 10.1126/scitranslmed.abj4011
Lipsman, N. et al. Blood-brain barrier opening in Alzheimer’s disease using MR-guided focused ultrasound. Nat. Commun. 9, 2336 (2018).
pubmed: 30046032
pmcid: 6060168
doi: 10.1038/s41467-018-04529-6
Idbaih, A. et al. Safety and feasibility of repeated and transient blood-brain barrier disruption by pulsed ultrasound in patients with recurrent glioblastoma. Clin. Cancer Res. 25, 3793–3801 (2019).
pubmed: 30890548
doi: 10.1158/1078-0432.CCR-18-3643
Carpentier, A. et al. Clinical trial of blood-brain barrier disruption by pulsed ultrasound. Sci. Transl. Med. 8, 343re2 (2016).
pubmed: 27306666
doi: 10.1126/scitranslmed.aaf6086
Sonabend, A. M. & Stupp, R. Overcoming the blood-brain barrier with an implantable ultrasound device. Clin. Cancer Res. 25, 3750–3752 (2019).
pubmed: 31076548
pmcid: 7057332
doi: 10.1158/1078-0432.CCR-19-0932
Gould, A. et al. Advances in blood-brain barrier disruption to facilitate drug delivery for infiltrative gliomas. Adv. Oncol. 3, 77–86 (2023).
pubmed: 37860079
pmcid: 10583833
doi: 10.1016/j.yao.2023.01.017
Carpentier, A. et al. Repeated blood–brain barrier opening with a nine-emitter implantable ultrasound device in combination with carboplatin in recurrent glioblastoma: a phase I/II clinical trial. Nat. Commun. 15, 1650 (2024).
pubmed: 38396134
pmcid: 10891097
doi: 10.1038/s41467-024-45818-7
Sonabend, A. M. et al. Repeated blood-brain barrier opening with an implantable ultrasound device for delivery of albumin-bound paclitaxel in patients with recurrent glioblastoma: a phase 1 trial. Lancet Oncol. 24, 509–522 (2023).
pubmed: 37142373
pmcid: 10256454
doi: 10.1016/S1470-2045(23)00112-2
Ahmed, M. H. et al. Low-Intensity pulsed ultrasound-mediated blood-brain barrier opening increases anti-programmed death-ligand 1 delivery and efficacy in Gl261 mouse model. Pharmaceutics 15, 455 (2023).
Rezai, A. R. et al. Ultrasound blood-brain barrier opening and aducanumab in Alzheimer’s Disease. N. Engl. J. Med. 390, 55–62 (2024).
pubmed: 38169490
doi: 10.1056/NEJMoa2308719
Reardon, D. A. et al. Effect of nivolumab vs bevacizumab in patients with recurrent glioblastoma: the checkmate 143 phase 3 randomized clinical trial. JAMA Oncol. 6, 1003–1010 (2020).
Omuro A., et al. Radiotherapy Combined With Nivolumab or Temozolomide for Newly Diagnosed Glioblastoma With Unmethylated MGMT Promoter: An International Randomized Phase 3 Trial. Neuro Oncol. Apr 14; https://doi.org/10.1093/neuonc/noac099 , (2022).
Lim M., et al. Phase 3 trial of chemoradiotherapy with temozolomide plus nivolumab or placebo for newly diagnosed glioblastoma with methylated MGMT promoter. Neuro Oncol. May 2; https://doi.org/10.1093/neuonc/noac116 , (2022).
Tewey, K. M., Rowe, T. C., Yang, L., Halligan, B. D. & Liu, L. F. Adriamycin-induced DNA damage mediated by mammalian DNA topoisomerase II. Science 226, 466–468 (1984).
pubmed: 6093249
doi: 10.1126/science.6093249
Kepp, O., Zitvogel, L. & Kroemer, G. Clinical evidence that immunogenic cell death sensitizes to PD-1/PD-L1 blockade. Oncoimmunology 8, e1637188 (2019).
pubmed: 31646079
pmcid: 6791413
doi: 10.1080/2162402X.2019.1637188
Casares, N. et al. Caspase-dependent immunogenicity of doxorubicin-induced tumor cell death. J. Exp. Med. 202, 1691–1701 (2005).
pubmed: 16365148
pmcid: 2212968
doi: 10.1084/jem.20050915
Ka, N. L., Lim, G. Y., Hwang, S., Kim, S. S. & Lee, M. O. IFI16 inhibits DNA repair that potentiates type-I interferon-induced antitumor effects in triple negative breast cancer. Cell Rep. 37, 110138 (2021).
pubmed: 34936865
doi: 10.1016/j.celrep.2021.110138
Luthra P., et al. Topoisomerase II Inhibitors induce DNA damage-dependent interferon responses circumventing ebola virus immune evasion. mBio. 04 04;8, https://doi.org/10.1128/mBio.00368-17 , (2017).
Wilkinson RDA, et al. Topoisomerase II inhibitors induce cGAS-STING dependent inflammation resulting in cytokine induction and immune checkpoint activation. bioRxiv. https://doi.org/10.1101/764662 , (2019).
Sistigu, A. et al. Cancer cell-autonomous contribution of type I interferon signaling to the efficacy of chemotherapy. Nat. Med. 20, 1301–1309 (2014).
pubmed: 25344738
doi: 10.1038/nm.3708
Ma, Y. et al. Contribution of IL-17-producing gamma delta T cells to the efficacy of anticancer chemotherapy. J. Exp. Med. 208, 491–503 (2011).
pubmed: 21383056
pmcid: 3058575
doi: 10.1084/jem.20100269
Mattarollo, S. R. et al. Pivotal role of innate and adaptive immunity in anthracycline chemotherapy of established tumors. Cancer Res. 71, 4809–4820 (2011).
pubmed: 21646474
doi: 10.1158/0008-5472.CAN-11-0753
Galluzzi, L., Humeau, J., Buqué, A., Zitvogel, L. & Kroemer, G. Immunostimulation with chemotherapy in the era of immune checkpoint inhibitors. Nat. Rev. Clin. Oncol. 17, 725–741 (2020).
pubmed: 32760014
doi: 10.1038/s41571-020-0413-z
Voorwerk, L. et al. Immune induction strategies in metastatic triple-negative breast cancer to enhance the sensitivity to PD-1 blockade: the TONIC trial. Nat. Med. 25, 920–928 (2019).
pubmed: 31086347
doi: 10.1038/s41591-019-0432-4
Portnow J., et al. Systemic Anti-PD-1 Immunotherapy Results in PD-1 Blockade on T Cells in the Cerebrospinal Fluid. JAMA Oncol. https://doi.org/10.1001/jamaoncol.2020.4508 , (2020).
Petereit, H. F. & Rubbert-Roth, A. Rituximab levels in cerebrospinal fluid of patients with neurological autoimmune disorders. Mult. Scler. 15, 189–192 (2009).
pubmed: 18971221
doi: 10.1177/1352458508098268
Stemmler, H. J. et al. Ratio of trastuzumab levels in serum and cerebrospinal fluid is altered in HER2-positive breast cancer patients with brain metastases and impairment of blood-brain barrier. Anticancer Drugs 18, 23–28 (2007).
pubmed: 17159499
doi: 10.1097/01.cad.0000236313.50833.ee
Alagkiozidis, I. et al. Increased immunogenicity of surviving tumor cells enables cooperation between liposomal doxorubicin and IL-18. J. Transl. Med. 7, 104 (2009).
pubmed: 20003308
pmcid: 2797002
doi: 10.1186/1479-5876-7-104
Takayama T., et al. Adjuvant Antitumor Immunity Contributes to the Overall Antitumor Effect of Pegylated Liposomal Doxorubicin (Doxil. Pharmaceutics. Oct 19;12, https://doi.org/10.3390/pharmaceutics12100990 (2020).
Yuan, J. et al. Single-cell transcriptome analysis of lineage diversity in high-grade glioma. Genome Med. 10, 57 (2018).
pubmed: 30041684
pmcid: 6058390
doi: 10.1186/s13073-018-0567-9
Hambardzumyan, D., Gutmann, D. H. & Kettenmann, H. The role of microglia and macrophages in glioma maintenance and progression. Nat. Neurosci. 19, 20–27 (2016).
pubmed: 26713745
pmcid: 4876023
doi: 10.1038/nn.4185
Arrieta V. A., et al. The Eclectic Nature of Glioma-Infiltrating Macrophages and Microglia. Int. J. Mol. Sci. 22 https://doi.org/10.3390/ijms222413382 (2021).
Garcia-Diaz, A. et al. Interferon receptor signaling pathways regulating PD-L1 and PD-L2 Expression. Cell Rep. 19, 1189–1201 (2017).
pubmed: 28494868
pmcid: 6420824
doi: 10.1016/j.celrep.2017.04.031
Schroder, K., Hertzog, P. J., Ravasi, T. & Hume, D. A. Interferon-gamma: an overview of signals, mechanisms and functions. J. Leukoc. Biol. 75, 163–189 (2004).
pubmed: 14525967
doi: 10.1189/jlb.0603252
Patnaik, A. et al. Phase I Study of Pembrolizumab (MK-3475; Anti-PD-1 Monoclonal Antibody) in patients with advanced solid tumors. Clin. Cancer Res. 21, 4286–4293 (2015).
pubmed: 25977344
doi: 10.1158/1078-0432.CCR-14-2607
Wendeln, A. C. et al. Innate immune memory in the brain shapes neurological disease hallmarks. Nature 556, 332–338 (2018).
pubmed: 29643512
pmcid: 6038912
doi: 10.1038/s41586-018-0023-4
Neftel, C. et al. An Integrative Model of Cellular States, Plasticity, and Genetics for Glioblastoma. Cell 178, 835–849.e21 (2019).
pubmed: 31327527
pmcid: 6703186
doi: 10.1016/j.cell.2019.06.024
Lyseng-Williamson, K. A., Duggan, S. T. & Keating, G. M. Pegylated liposomal doxorubicin: a guide to its use in various malignancies. BioDrugs 27, 533–540 (2013).
pubmed: 24018470
doi: 10.1007/s40259-013-0070-1
Kim, C. et al. Closed-loop trans-skull ultrasound hyperthermia leads to improved drug delivery from thermosensitive drugs and promotes changes in vascular transport dynamics in brain tumors. Theranostics 11, 7276–7293 (2021).
pubmed: 34158850
pmcid: 8210606
doi: 10.7150/thno.54630
Kovacs, Z. et al. Prolonged survival upon ultrasound-enhanced doxorubicin delivery in two syngenic glioblastoma mouse models. J. Control Release 187, 74–82 (2014).
pubmed: 24878186
doi: 10.1016/j.jconrel.2014.05.033
Mainprize, T. et al. Blood-brain barrier opening in primary brain tumors with non-invasive mr-guided focused ultrasound: a clinical safety and feasibility study. Sci. Rep. 9, 321 (2019).
pubmed: 30674905
pmcid: 6344541
doi: 10.1038/s41598-018-36340-0
Woroniecka, K. et al. T-cell exhaustion signatures vary with tumor type and are severe in glioblastoma. Clin. Cancer Res. 24, 4175–4186 (2018).
pubmed: 29437767
pmcid: 6081269
doi: 10.1158/1078-0432.CCR-17-1846
Pang, B. et al. Drug-induced histone eviction from open chromatin contributes to the chemotherapeutic effects of doxorubicin. Nat. Commun. 4, 1908 (2013).
pubmed: 23715267
doi: 10.1038/ncomms2921
Yang, F., Kemp, C. J. & Henikoff, S. Doxorubicin enhances nucleosome turnover around promoters. Curr. Biol. 23, 782–787 (2013).
pubmed: 23602475
pmcid: 3678987
doi: 10.1016/j.cub.2013.03.043
Arrieta V. A., et al. ERK1/2 phosphorylation predicts survival following anti-PD-1 immunotherapy in recurrent glioblastoma. Nat Cancer. https://doi.org/10.1038/s43018-021-00260-2 (2021).
Zhao, J. et al. Immune and genomic correlates of response to anti-PD-1 immunotherapy in glioblastoma. Nat. Med. 25, 462–469 (2019).
pubmed: 30742119
pmcid: 6810613
doi: 10.1038/s41591-019-0349-y
Johanns, T. M. et al. Immunogenomics of Hypermutated Glioblastoma: A Patient with Germline POLE Deficiency Treated with Checkpoint Blockade Immunotherapy. Cancer Discov. 6, 1230–1236 (2016).
pubmed: 27683556
pmcid: 5140283
doi: 10.1158/2159-8290.CD-16-0575
Arrieta V. A., et al. ERK1/2 Phosphorylation Predicts Survival in Recurrent Glioblastoma Following Intracerebral and Adjuvant PD-1/CTLA-4 Immunotherapy: A REMARK-Guided Analysis. Clin. Cancer Res. Nov 08; https://doi.org/10.1158/1078-0432.CCR-23-1889 (2023).
Arrieta V. A., et al. Immune checkpoint blockade in glioblastoma: from tumor heterogeneity to personalized treatment. J Clin Invest. Jan 17 133, https://doi.org/10.1172/JCI163447 , (2023).
Yost, K. E. et al. Clonal replacement of tumor-specific T cells following PD-1 blockade. Nat. Med. 25, 1251–1259 (2019).
pubmed: 31359002
pmcid: 6689255
doi: 10.1038/s41591-019-0522-3
Rodig S. J., et al. MHC proteins confer differential sensitivity to CTLA-4 and PD-1 blockade in untreated metastatic melanoma. Sci Transl Med. 07 18 10 https://doi.org/10.1126/scitranslmed.aar3342 (2018).
Johnson, D. B. et al. Melanoma-specific MHC-II expression represents a tumour-autonomous phenotype and predicts response to anti-PD-1/PD-L1 therapy. Nat. Commun. 7, 10582 (2016).
pubmed: 26822383
pmcid: 4740184
doi: 10.1038/ncomms10582
Paulson, K. G. et al. Acquired cancer resistance to combination immunotherapy from transcriptional loss of class I HLA. Nat. Commun. 9, 3868 (2018).
pubmed: 30250229
pmcid: 6155241
doi: 10.1038/s41467-018-06300-3
Verma, V. et al. PD-1 blockade in subprimed CD8 cells induces dysfunctional PD-1
pubmed: 31358999
pmcid: 7472661
doi: 10.1038/s41590-019-0441-y
Lee, E. K. et al. Combined pembrolizumab and pegylated liposomal doxorubicin in platinum resistant ovarian cancer: A phase 2 clinical trial. Gynecol. Oncol. 159, 72–78 (2020).
pubmed: 32771276
doi: 10.1016/j.ygyno.2020.07.028
Kyte, J. A. et al. ICON: a randomized phase IIb study evaluating immunogenic chemotherapy combined with ipilimumab and nivolumab in patients with metastatic hormone receptor positive breast cancer. J. Transl. Med. 18, 269 (2020).
pubmed: 32620163
pmcid: 7333428
doi: 10.1186/s12967-020-02421-w
Mittendorf, E. A. et al. Neoadjuvant atezolizumab in combination with sequential nab-paclitaxel and anthracycline-based chemotherapy versus placebo and chemotherapy in patients with early-stage triple-negative breast cancer (IMpassion031): a randomised, double-blind, phase 3 trial. Lancet 396, 1090–1100 (2020).
pubmed: 32966830
doi: 10.1016/S0140-6736(20)31953-X
Pollack, S. M. et al. Assessment of doxorubicin and pembrolizumab in patients with advanced anthracycline-naive sarcoma: a phase 1/2 nonrandomized clinical trial. JAMA Oncol. 6, 1778–1782 (2020).
pubmed: 32910151
doi: 10.1001/jamaoncol.2020.3689
Iwai, Y. et al. Involvement of PD-L1 on tumor cells in the escape from host immune system and tumor immunotherapy by PD-L1 blockade. Proc. Natl Acad. Sci. USA. 99, 12293–12297 (2002).
pubmed: 12218188
pmcid: 129438
doi: 10.1073/pnas.192461099
Arrieta, V. A. et al. Can patient selection and neoadjuvant administration resuscitate PD-1 inhibitors for glioblastoma? J. Neurosurg. 132, 1667–1672 (2019).
McGrail, D. J. et al. Replication stress response defects are associated with response to immune checkpoint blockade in nonhypermutated cancers. Sci. Transl. Med. 13, eabe6201 (2021).
pubmed: 34705519
pmcid: 8577990
doi: 10.1126/scitranslmed.abe6201
Gromeier, M. et al. Very low mutation burden is a feature of inflamed recurrent glioblastomas responsive to cancer immunotherapy. Nat. Commun. 12, 352 (2021).
pubmed: 33441554
pmcid: 7806846
doi: 10.1038/s41467-020-20469-6
Gill, B. J. et al. MRI-localized biopsies reveal subtype-specific differences in molecular and cellular composition at the margins of glioblastoma. Proc. Natl Acad. Sci. Usa. 111, 12550–12555 (2014).
pubmed: 25114226
pmcid: 4151734
doi: 10.1073/pnas.1405839111
Chang, P. D. et al. A multiparametric model for mapping cellularity in glioblastoma using radiographically localized biopsies. AJNR Am. J. Neuroradiol. 38, 890–898 (2017).
pubmed: 28255030
pmcid: 7960397
doi: 10.3174/ajnr.A5112
Amir el-, A. D. et al. viSNE enables visualization of high dimensional single-cell data and reveals phenotypic heterogeneity of leukemia. Nat. Biotechnol. 31, 545–552 (2013).
doi: 10.1038/nbt.2594
Lee-Chang, C. et al. Myeloid-derived suppressive cells promote b cell-mediated immunosuppression via transfer of pd-l1 in glioblastoma. Cancer Immunol. Res. 7, 1928–1943 (2019).
pubmed: 31530559
pmcid: 6891201
doi: 10.1158/2326-6066.CIR-19-0240
Chen, H. et al. Cytofkit: a bioconductor package for an integrated mass cytometry data analysis pipeline. PLoS Comput Biol. 12, e1005112 (2016).
pubmed: 27662185
pmcid: 5035035
doi: 10.1371/journal.pcbi.1005112
Nair, A. B. & Jacob, S. A simple practice guide for dose conversion between animals and human. J. Basic Clin. Pharm. 7, 27–31 (2016).
pubmed: 27057123
pmcid: 4804402
doi: 10.4103/0976-0105.177703
Bates, D. M., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015).
doi: 10.18637/jss.v067.i01