Hands-on with horizontally scaling Python for production By MAJ Brent Stone, Cyber Solutions Development, Army Cyber Command This was a hands-on Python programming workshop. Python can be challenging to use in production when "real-world" workloads involving megabits per second (Mbps) of streaming data or terabytes of stored data are involved. The Global Interpreter Lock (GIL) means that a Python interpreter is effectively single-threaded and can't take advantage of modern processors' capacity for parallel computation. Laterally scaling workloads across many Python interpreters is one of
Hands-on with horizontally scaling Python for production By MAJ Brent Stone, Cyber Solutions Development, Army Cyber Command This was a hands-on Python programming workshop. Python can be challenging to use in production when "real-world" workloads involving megabits per second (Mbps) of streaming data or terabytes of stored data are involved. The Global Interpreter Lock (GIL) means that a Python interpreter is effectively single-threaded and can't take advantage of modern processors' capacity for parallel computation. Laterally scaling workloads across many Python interpreters is one of the most viable workarounds to the shortcomings of the GIL. This workshop will introduce you to two leading frameworks for doing this: Celery and Dask (via Prefect). This workshop will walk through establishing an Extract, Transform, Load (ETL) pipeline in each framework which reads and writes from Redis and a MinIO locally hosted S3 bucket.
Size: 6720px × 4480px
Photo credit: © Operation 2024 / Alamy / Afripics
License: Licensed
Model Released: No
Keywords: army, avengercon, cyber, cybersecurity, hackers
