pyspark-bbn

Preface

pyspark-bbn is a scalable, massively parallel processing MPP framework for learning structures and parameters of Bayesian Belief Networks BBNs. The API leverages Apache Spark for high-performance HPC computing needs. While there are papers and approaches to scalable learning of BBNs, to our knowledge, pyspark-bbn is the first of its kind to leverage Apache Spark. We have implemented several structure learning algorithms for both discrete and gaussian variables. Please contact us at info@rocketvector.io to discuss our products and services.

Contents

• Introduction
  • The Art of the Possible
  • Simple, unified and effective
  • Products and Service

Structure Learning

• Constraint-Based
  • Load data
  • Discrete data
  • Constraint-based
    • Naive Bayes
    • Tree augmented network
    • BN augmented naive Bayes
    • Maximum weight spanning tree
    • Three phase dependency analysis
    • PC algorithm
• Search-and-Scoring
  • Load data
  • Genetic algorithm

Parameter Learning

• Parameter Learning
  • Load data
  • Structure learning
  • Parameter learning

BBN

• Full Example
  • Load data
  • Structure learning
  • Parameter learning
  • BBN
  • Inference
• Cloud
  • GCP, Dataproc
  • AWS, Spark Processing Job
  • Azure, Machine Learning

Use Cases

• Diabetes
  • Load data
  • Split data
  • Spark
  • Naive Bayes (NB)
    • NB, structure learning
    • NB, Parameter learning
    • NB, Posteriors
    • NB, lift
  • Tree
  • Tree, structure learning
    • Tree, parameter learning
  • Tree, posteriors
    • Tree, lift
  • Validation
  • How does age impact diabetes?
  • How does gender impact diabetes?
  • How does BMI impact diabetes?
  • How does heart disease impact diabetes?
  • How does smoking status impact diabetes?
• Housing Prices
  • Spark
  • Learning
  • Lift
• Congressional Support
  • Initialize
  • BBN
  • Lift
  • Average Causal Effect (ACE )
    • Health
    • International affairs
    • Government operations and politics
    • Armed forces and national security
    • Crime and law enforcement
    • Abortion
• Congressional Influence
  • Load data
  • Learning
  • Influence
    • Nancy Pelosi
    • Alexandria Ocasio-Cortez
    • Kevin McCarthy
    • CA influencers
    • NY influencers
    • VA influencers
• Congressional Influence - All
  • Learning
  • Structure
  • Influence
  • Cross-party influence
  • Within and between state influence
  • Largest lift, different districts
  • Largest lift, different parties
  • Largest lift, different states
  • Largest lift, different district, party, state

API Documentation

• pyspark-bbn
  • Subpackages
    • Learning from Discrete Data
      • Data
      • Structure Learning
        • Constraint-Based
        • Search-and-Scoring
      • Parameter Learning
      • Utilities
    • Learning from Continuous Data
      • Data
      • Structure Learning
        • Constraint-Based
        • Search-and-Scoring

About

• Website

• Facebook

• Twitter

• Instagram

• YouTube

• LinkedIn

Copyright

Software

This work is copyrighted by Rocket Vector, 2020.

Art

Copyright 2020 Daytchia Vang

Citation

@misc{rocketvector_sdk_2020,
title={pyspark-bbn},
url={https://sdk.rocketvector.io},
author={Rocket Vector},
year={2020},
month={May}}