Package: SBC 0.5.0.0

Angie.H Moon

SBC: Simulation Based Calibration for Bayesian models

SBC helps perform Simulation Based Calibration on Bayesian models. SBC lets you check for bugs in your model code and/or algorithm that fits the model. SBC focuses on models built with 'Stan' <https://mc-stan.org>, but can support other modelling languages as well.

Authors:Martin Modrák [aut], Shinyoung Kim [aut], Angie.H Moon [aut, cre], Teemu Säilynoja [aut], Luna Fazio [ctb], Timo Stenz [ctb]

SBC_0.5.0.0.tar.gz
SBC_0.5.0.0.zip(r-4.6)SBC_0.5.0.0.zip(r-4.5)SBC_0.5.0.0.zip(r-4.4)
SBC_0.5.0.0.tgz(r-4.6-any)SBC_0.5.0.0.tgz(r-4.5-any)
SBC_0.5.0.0.tar.gz(r-4.6-any)SBC_0.5.0.0.tar.gz(r-4.5-any)
SBC_0.5.0.0.tgz(r-4.5-emscripten)
SBC.pdf |SBC.html✨
SBC/json (API)

# Install 'SBC' in R:
install.packages('SBC', repos = c('https://community.r-multiverse.org', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/hyunjimoon/sbc/issues

Pkgdown/docs site:https://hyunjimoon.github.io

On CRAN:

Conda:

diagnosticssimulation-based-inference

6.33 score 62 stars 138 scripts 116 exports 47 dependencies

Last updated from:e40a320c33 (on v0.5.0). Checks:8 ERROR, 1 OK. Indexed: no.

TargetResultTimeFilesSyslog
linux-devel-x86_64ERROR256
source / vignettesERROR3294
linux-release-x86_64ERROR203
macos-devel-arm64ERROR992
macos-release-arm64ERROR976
windows-develERROR133
windows-releaseERROR138
windows-oldrelERROR182
wasm-releaseOK179

Exports:attribute_presentattribute_present_statsbinary_calibration_from_bpbinary_miscalibrationbinary_probabilities_from_statsbinary_var_attributebind_datasetsbind_derived_quantitiesbind_generated_quantitiesbind_resultsbrier_resampling_pbrier_resampling_testcached_fit_filenamecalculate_prior_sdcalculate_ranks_draws_matrixcalculate_sds_draws_matrixcheck_all_SBC_diagnosticscombine_all_variablescombine_array_elementscombine_var_attributescompute_dquantscompute_gen_quantscompute_resultscompute_SBCcount_diagnosticdata_for_ecdf_plotsdefault_chunk_sizedefault_cores_per_fitdefault_diagnosticdefault_diagnostics_typesderived_quantitiesdiagnostic_typesdquants_var_attributesdraws_rvars_to_standatadraws_rvars_to_standata_singleempirical_coverageextract_attribute_arguments_statsgaffke_cigaffke_pgaffke_testgenerate_datasetsgenerated_quantitiesget_all_diagnostic_messagesget_diagnostic_messages_singleget_stats_for_submodelhidden_var_attributeinf_valid_var_attributelogical_diagnosticmiscalibration_resampling_pmiscalibration_resampling_testna_valid_var_attributenumeric_diagnosticplot_binary_calibrationplot_binary_calibration_diffplot_contractionplot_coverageplot_coverage_diffplot_ecdfplot_ecdf_diffplot_rank_histplot_sim_estimatedpossibly_constant_var_attributeposterior_cdf_from_model_probsrecompute_SBC_statisticsrecompute_statisticsremove_attribute_from_statsSBC_backend_bridgesamplingSBC_backend_brmsSBC_backend_brms_from_generatorSBC_backend_cachedSBC_backend_cmdstan_sampleSBC_backend_cmdstan_variationalSBC_backend_default_thin_ranksSBC_backend_extractBF_comparisonSBC_backend_functionSBC_backend_hash_for_cacheSBC_backend_iid_drawsSBC_backend_lmBFSBC_backend_mockSBC_backend_mock_rngSBC_backend_postprocess_cached_fitSBC_backend_preprocess_fit_for_cacheSBC_backend_rjagsSBC_backend_rstan_optimizingSBC_backend_rstan_sampleSBC_datasetsSBC_datasets_for_bfSBC_example_backendSBC_example_generatorSBC_example_resultsSBC_fitSBC_fit_specific_dquantsSBC_fit_to_BFBayesFactorSBC_fit_to_bridge_samplerSBC_fit_to_diagnosticsSBC_fit_to_draws_matrixSBC_generator_brmsSBC_generator_customSBC_generator_functionSBC_posterior_cdfSBC_print_example_modelSBC_resultsSBC_results_diagnostic_messagesSBC_results_diagnosticsSBC_statistics_from_single_fitskip_diagnosticsplit_SBC_results_for_bfsubmodel_diagnosticsubmodel_var_attributetop_model_dquantsvalidate_derived_quantitiesvalidate_generated_quantitiesvalidate_SBC_datasetsvalidate_SBC_resultsvalidate_var_attributesvar_attributes

Dependencies:abindbackportscachemcheckmateclicodetoolscpp11digestdistributionaldplyrfarverfastmapfuturefuture.applygenericsggplot2globalsgluegtableisobandlabelinglifecyclelistenvmagrittrmatrixStatsmemoisenumDerivparallellypillarpkgconfigposteriorpurrrR6RColorBrewerrlangS7scalesstringistringrtensorAtibbletidyrtidyselectutf8vctrsviridisLitewithr

Readme and manuals

Help Manual

Help pageTopics
Subset an 'SBC_datasets' object.[.SBC_datasets
Subset the results.[.SBC_results
Get attribute presence for a vector of variable names.attribute_present
Check an attribute in the $stats element of the results for presence of an attribute.attribute_present_stats
Binary prediction calibration computation and visualisation.binary_calibration_from_bp plot_binary_calibration plot_binary_calibration.data.frame plot_binary_calibration.SBC_results plot_binary_calibration_diff plot_binary_calibration_diff.data.frame plot_binary_calibration_diff.SBC_results
Extract binary probabilities from SBC stats.binary_probabilities_from_stats
Combine multiple datasets together.bind_datasets
Combine two lists of derived quantitiesbind_derived_quantities
Combine two sets globals for use in derived quantities or backendbind_globals
Combine multiple SBC results together.bind_results
Binary calibration testsbinary_miscalibration brier_resampling_p brier_resampling_test miscalibration_resampling_p miscalibration_resampling_test
Provide a file where a cached result of a fit will be/was stored.cached_fit_filename
Calculate prior standard deviation of a datasetcalculate_prior_sd
Calculate ranks given variable values within a posterior distribution.calculate_ranks_draws_matrix
Check diagnostics for an SBC results object.check_all_SBC_diagnostics
Helper functions to be passed to ECDF-plots to combine variables in a single panel.combine_all_variables combine_array_elements
Combine two named lists and overwrite elements with the same name using the value from args2combine_args
Combine multiple sets of variable attributes.combine_var_attributes
Compute derived quantities based on given data and posterior draws.compute_dquants
Fit datasets and evaluate diagnostics and SBC metrics.compute_SBC
Maybe not export in the end? Useful for debuggingdata_for_ecdf_plots
Determines the default chunk size.default_chunk_size
Determines the default cores per single fit.default_cores_per_fit
Create a definition of derived quantities evaluated in R.derived_quantities
Get the 'var_attributes()' object associated with the derived quantitiesdquants_var_attributes
Compute observed coverage of posterior credible intervals.empirical_coverage
Extract arguments of a given attribute from the $stats element of the resultsextract_attribute_arguments_stats
Non-parametric test for the mean of a bounded variable.gaffke_ci gaffke_p gaffke_test
Generate datasets.generate_datasets
Extract stats for a submodel from a Bayes factor resultsget_stats_for_submodel
Guess the number of bins for 'plot_rank_hist()'.guess_rank_hist_bins
Prior/posterior contraction plot.plot_contraction plot_contraction.data.frame
Plot the observed coverage and its uncertainty.plot_coverage plot_coverage_diff
Plot the ECDF-based plots.plot_ecdf plot_ecdf_diff
Plot rank histogram of an SBC results.plot_rank_hist plot_rank_hist.data.frame
Plot the simulated "true" values versus posterior estimatesplot_sim_estimated plot_sim_estimated.data.frame
Custom rbind implementation maintainig information about submodelsrbind.SBC_bridgesampling_diagnostics
Recompute SBC statistics without refitting models.recompute_SBC_statistics
Remove an attribute from a 'data.frame' column with attributes.remove_attribute_from_stats
Backend using bridgesampling to do Bayes factor calculation (and Bayesian Model Averaging) over two candidate models.SBC_backend_bridgesampling
Build a backend based on the 'brms' package.SBC_backend_brms
Build a brms backend, reusing the compiled model from a previously created 'SBC_generator_brms' object.SBC_backend_brms_from_generator
A backend that wraps another backend with a caching scheme.SBC_backend_cached
Backend based on sampling via 'cmdstanr'.SBC_backend_cmdstan_sample
Backend based on variational approximation via 'cmdstanr'.SBC_backend_cmdstan_variational
S3 generic to get backend-specific default thinning for rank computation.SBC_backend_default_thin_ranks SBC_backend_default_thin_ranks.default
A backend for Bayes factor workflow with BayesFactor::extractBF.SBC_backend_extractBF_comparison
A backend that just calls a function.SBC_backend_function
Get hash used to identify cached results.SBC_backend_hash_for_cache
S3 generic to let backends signal that they produced independent draws.SBC_backend_iid_draws SBC_backend_iid_draws.default
A backend using 'BayesFactor::lmBF()'SBC_backend_lmBF
A mock backend.SBC_backend_mock
Allows the backend to do any pre-/post- processing on a fit stored to / loaded from cache.SBC_backend_postprocess_cached_fit SBC_backend_preprocess_fit_for_cache
Create a JAGS backend using 'rjags'SBC_backend_rjags
SBC backend using the 'optimizing' method from 'rstan'.SBC_backend_rstan_optimizing
SBC backend using the 'sampling' method from 'rstan'.SBC_backend_rstan_sample
Create new 'SBC_datasets' object.SBC_datasets
Combine two or more datasets to check Bayes factor computation.SBC_datasets_for_bf
Construct a backend to be used in the examples.SBC_example_backend
Construct a generator used in the examples.SBC_example_generator
Combine an example backend with an example generator to provide full results that can be used to test other functions in the package.SBC_example_results
S3 generic using backend to fit a model to data.SBC_fit
A fits need to implement this S3 generic to be useful for the 'SBC_backend_extractBF_comparison()' backend.SBC_fit_to_BFBayesFactor
Convert a fit for a given backend into a bridge sampler object.SBC_fit_to_bridge_sampler
S3 generic to get backend-specific diagnostics.SBC_fit_to_diagnostics
S3 generic converting a fitted model to a 'draws_matrix' object.SBC_fit_to_draws_matrix SBC_fit_to_draws_matrix.default
Create a brms generator.SBC_generator_brms
Wrap a function the creates a complete dataset.SBC_generator_custom
Generate datasets via a function that creates a single dataset.SBC_generator_function
Use a fitted model to obtain a posterior CDF at a given point.SBC_posterior_cdf SBC_posterior_cdf.default
Print the Stan code of a model used in the examples.SBC_print_example_model
Create an 'SBC_results' objectSBC_results
Recompute SBC statistics given a single fit.SBC_statistics_from_single_fit
Custom select implementation maintaining information about submodelsselect.SBC_bridgesampling_diagnostics
Split stats from SBC of Bayes factor into stats for the individual submodels.split_SBC_results_for_bf
Validate a definition of derived quantities evaluated in R.validate_derived_quantities
Predefined constants to use for variable attributes recognized by SBC.binary_var_attribute hidden_var_attribute inf_valid_var_attribute na_valid_var_attribute possibly_constant_var_attribute submodel_var_attribute validate_var_attributes var_attributes