SlideShare a Scribd company logo
1 of 22
Download to read offline
Clear Linux* OS
Architecture Overview
Patrick McCarty - Software Engineer
*Other names and brands may be claimed as the property of others
© 2018 Intel Corporation. Intel, the Intel logo, Intel Inside, the Intel Inside logo, Intel Experience What’s Inside, The Intel Experience What’s Inside logo, and Xeon are trademarks of Intel Corporation in the U.S.
and/or other countries. *Other names and brands may be claimed as the property of others.
Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer.
Intel processors of the same SKU may vary in frequency or power as a result of natural variability in the production process.
For more complete information about performance and benchmark results, visit www.intel.com/benchmarks.
The cost reduction scenarios described are intended to enable you to get a better understanding of how the purchase of a given Intel based product, combined with a number of situation-specific variables, might
affect future costs and savings. Circumstances will vary and there may be unaccounted-for costs related to the use and deployment of a given product. Nothing in this document should be interpreted as either a
promise of or contract for a given level of costs or cost reduction.
Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate.
Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2,
SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-
dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the
applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision #20110804.
No computer system can be absolutely secure.
Intel® Advanced Vector Extensions (Intel® AVX)* provides higher throughput to certain processor operations. Due to varying processor power characteristics, utilizing AVX instructions may cause a) some parts to
operate at less than the rated frequency and b) some parts with Intel® Turbo Boost Technology 2.0 to not achieve any or maximum turbo frequencies. Performance varies depending on hardware, software, and
system configuration and you can learn more at http://www.intel.com/go/turbo.
Available on select Intel® processors. Requires an Intel® HT Technology-enabled system. Your performance varies depending on the specific hardware and software you use. Learn more by visiting
http://www.intel.com/info/hyperthreading.
§ Configurations: The testing was done on Based on fourth-generation Intel Xeon E5-2699 v4 @2.20 GHz processor with 22 cores, 55 MB LLC and 62 GB memory 16 1G hugepages. The testing was conducted in OPNFV
Pharos testbed on Pod 12 by VSPERF community engineers
Intel, the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others.
© Intel Corporation
Legal Disclaimer
Agenda
● Clear Linux* OS Overview
● Performance optimizations
● Use-case focused bundles
● Stateless OS design
● Telemetry
● Updates
*Other names and brands may be claimed as the property of others
Clear Linux* OS Overview
● Optimized for IA
● Rolling release distribution
● Average of 9 releases per week
● Developer-focused
*Other names and brands may be claimed as the property of others
Performance Optimizations
● Optimize the entire stack
● Compiler flags
○ Westmere baseline
○ Haswell tuned
● Optimized libraries selected at runtime based
on available CPU features
● Performance patches to packages
● Example optimized package:
https://github.com/clearlinux-
pkgs/opencv/blob/master/opencv.spec
Program using OpenCV*
AVX2-enabled CPU
dynamic linker
libopencv_*.so (base)
libopencv_*.so (avx2)
*Other names and brands may be claimed as the property of others
Use-Case Focused Bundles
● Bundles provide use-case driven
functionality to end user
● Dependencies resolved at build time on
server, not at install or runtime
● Similar to package groups in other distros
● Vertically vs horizontally integrated os-core
os-core-update
network-basic
webserver openssl
python-basic
application-server
kvm-host
ansible iproute2
virt-manager
scm-server
cloud-control
*Other names and brands may be claimed as the property of others
TRADITIONAL OS
User Data
System Configuration
Operating System
CLEAR LINUX* OS
User Data
System Configuration
Operating System
Stateless
● OS provides functional and secure default
configuration in /usr
● Defaults can be overridden or modified in /etc
and the home directory
● Wiping /etc and /var performs a "factory reset",
restoring OS default configs
*Other names and brands may be claimed as the property of others
Stateless – example
● Default telemetrics.conf from operating system in /usr
record_expiry=1200
spool_max_size=5120
spool_process_time=900
rate_limit_enabled=true
record_burst_limit=1000
record_window_length=15
Stateless – example
● Default telemetrics.conf from operating system in /usr
● Custom configuration in /etc
record_expiry=1200
spool_max_size=5120
spool_process_time=900
rate_limit_enabled=true
record_burst_limit=1000
record_window_length=15
record_expiry=1200
spool_max_size=5120
spool_process_time=900
rate_limit_enabled=false
record_burst_limit=1000
record_window_length=15
Stateless – example
● Default telemetrics.conf from operating system in /usr
● Custom configuration in /etc
record_expiry=1200
spool_max_size=5120
spool_process_time=900
rate_limit_enabled=true
record_burst_limit=1000
record_window_length=15
record_expiry=1200
spool_max_size=5120
spool_process_time=900
rate_limit_enabled=false
record_burst_limit=1000
record_window_length=15
Telemetry
● Opt-in telemetry solution
● Lightweight client service
● Client-side probes send records to help debug software anomalies.
● Probes avoid collecting personally identifiable information and records
comply with Privacy Policy*.
● Records are analyzed and displayed in a developer-oriented format on the
telemetry server.
* https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html
Probe
Probe
Probe
…
telemprobd
libtelemetry
telempostd
Server
Telemetry – Client architecture
Updating
● All installed bundles are updated at once
○ Entire system update (one OS version)
○ QA is done on the entire OS release at once
● Proportional updates
● Auto-update on by default
Update content created by mixer tool
Upstream
Sources
Bundle
definitions
Bundle A
data
Bundle B
data
Bundle C
data
Full chroot
Update Creator
Update
Creator
Update
Artifacts
Swupd
clients...
swupd
clients...
Mixing
Mixing – Update artifacts
Manifests
MANIFEST 24 # OS tooling/content format
version: 21260 # OS Version this manifest describes
previous: 21220 # Previous change to this manifest at this OS version
filecount: 13624 # Number of files in the manifest
timestamp: 1520706949 # Epoch of creation
contentsize: 811403622 # Size, in bytes, of this bundle (not accounting for included bundles)
includes: os-core # Bundle included by this bundle
F... 0437fc1556fdfe08ee8cfa492094e5c11a86b7b793213767d4f5697d9b437b36 21080 /usr/bin/c_hash
F... 4fdebd92c2ad33ad063c8de973b4eafa35d800ff70abe75644172ae6d0b81436 21080 /usr/bin/corelist
< 13622 more entries >
Manifest of Manifests (MoM)
M... 39be958b03625d0507222996f167de279bc2edaec9a1ff45a86f3cdfac83ca6a 21080 desktop-autostart
M... 3ac656e9bdb43871f5345cf71c866a67a58d3ce0a2a085efb8e703be4dd3d753 21080 desktop-locales
M... 1dbd2354eb2cbf47a871a4d70fc5cee0dc0e6df2c940b03ab6d5ac2edbad594d 21080 dhcp-server
Manifest.MoM signed, verification cascades through SHA256 hashes
hashes verifiedhashes verified
Manifest.MoM
Manifest.MoM.sig
Manifest.os-core
Manifest.go-basic
Manifest.shells
Manifest.editors
Manifest.desktop
...
/usr
/usr/bin/bash
/usr/bin/cat
/usr/bin/chmod
/usr/bin/rm
...
/usr/bin/chown
/usr/bin/chroot
Mixing – Security
Mixing – Update artifacts
● Packs
○ Delta-packs (from version x to y, content difference between versions)
Binary deltas
○ Zero-packs (from version 0, complete content of bundle)
● Full files (for fallback)
○ Compressed full files available for download if pack download/extraction fails
NON-ATOMIC
ms-s durationATOMIC
Download +
verify MoM
Download +
extract packs
Apply delta
files and stage
Verify pack
contents with
manifests
Rename
staged to final
Update
Artifacts
Updating – Client Operation
Clear Linux
bundles and
content
Mixer Update
Artifacts
swupd
clients...
User bundles and
content
Creating Custom Mixes
Useful for teams that want to provide their own content on top of Clear
Linux* OS content for development, testing, etc.
*Other names and brands may be claimed as the property of others
mixin
Useful for individual users that want
to add their own content
User adds
package Mixer Local
Artifacts
Upstream
Artifacts
Merge
swupd
client
Side-loading Custom Content
Clear Linux* OS
● Rolling release security updates
● Stateless OS design
● Performance focused
● Use-case optimized bundles
● Fast, secure, and reliable updates
*Other names and brands may be claimed as the property of others
Contact details
Patrick McCarty
pmccarty on #clearlinux (freenode)
More resources:
Project site: clearlinux.org
Forum: community.clearlinux.org
Git repos:
github.com/clearlinux
github.com/clearlinux-pkgs

More Related Content

What's hot

Linux Kernel and Driver Development Training
Linux Kernel and Driver Development TrainingLinux Kernel and Driver Development Training
Linux Kernel and Driver Development TrainingStephan Cadene
 
IT Essentials (Version 7.0) - ITE Chapter 12 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 12 Exam AnswersIT Essentials (Version 7.0) - ITE Chapter 12 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 12 Exam AnswersITExamAnswers.net
 
32bit UEFIマシンで遊ぶ
32bit UEFIマシンで遊ぶ32bit UEFIマシンで遊ぶ
32bit UEFIマシンで遊ぶshimadah
 
IT Essentials (Version 7.0) - ITE Chapter 10 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 10 Exam AnswersIT Essentials (Version 7.0) - ITE Chapter 10 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 10 Exam AnswersITExamAnswers.net
 
IT Essentials (Version 7.0) - ITE Chapter 13 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 13 Exam AnswersIT Essentials (Version 7.0) - ITE Chapter 13 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 13 Exam AnswersITExamAnswers.net
 
UEFI時代のブートローダ
UEFI時代のブートローダUEFI時代のブートローダ
UEFI時代のブートローダTakuya ASADA
 
Linux booting Process
Linux booting ProcessLinux booting Process
Linux booting ProcessGaurav Sharma
 
OpenWrt From Top to Bottom
OpenWrt From Top to BottomOpenWrt From Top to Bottom
OpenWrt From Top to BottomKernel TLV
 
Getting Started With Linux Administration
Getting Started With Linux AdministrationGetting Started With Linux Administration
Getting Started With Linux AdministrationEdureka!
 
Kali linux
Kali linux Kali linux
Kali linux Fa6ma_
 
Intoduction to VirtualBox English
Intoduction to VirtualBox EnglishIntoduction to VirtualBox English
Intoduction to VirtualBox EnglishKichiemon Adachi
 
Redesigning Xen Memory Sharing (Grant) Mechanism
Redesigning Xen Memory Sharing (Grant) MechanismRedesigning Xen Memory Sharing (Grant) Mechanism
Redesigning Xen Memory Sharing (Grant) MechanismThe Linux Foundation
 

What's hot (20)

Linux Kernel and Driver Development Training
Linux Kernel and Driver Development TrainingLinux Kernel and Driver Development Training
Linux Kernel and Driver Development Training
 
Linux
LinuxLinux
Linux
 
IT Essentials (Version 7.0) - ITE Chapter 12 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 12 Exam AnswersIT Essentials (Version 7.0) - ITE Chapter 12 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 12 Exam Answers
 
Kali linux
Kali linuxKali linux
Kali linux
 
32bit UEFIマシンで遊ぶ
32bit UEFIマシンで遊ぶ32bit UEFIマシンで遊ぶ
32bit UEFIマシンで遊ぶ
 
IT Essentials (Version 7.0) - ITE Chapter 10 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 10 Exam AnswersIT Essentials (Version 7.0) - ITE Chapter 10 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 10 Exam Answers
 
Linux os and its features
Linux os and its featuresLinux os and its features
Linux os and its features
 
kali linux.pptx
kali linux.pptxkali linux.pptx
kali linux.pptx
 
IT Essentials (Version 7.0) - ITE Chapter 13 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 13 Exam AnswersIT Essentials (Version 7.0) - ITE Chapter 13 Exam Answers
IT Essentials (Version 7.0) - ITE Chapter 13 Exam Answers
 
UEFI時代のブートローダ
UEFI時代のブートローダUEFI時代のブートローダ
UEFI時代のブートローダ
 
Linux booting Process
Linux booting ProcessLinux booting Process
Linux booting Process
 
OpenWrt From Top to Bottom
OpenWrt From Top to BottomOpenWrt From Top to Bottom
OpenWrt From Top to Bottom
 
Platform Observability and Infrastructure Closed Loops
Platform Observability and Infrastructure Closed LoopsPlatform Observability and Infrastructure Closed Loops
Platform Observability and Infrastructure Closed Loops
 
Linux - Introductions to Linux Operating System
Linux - Introductions to Linux Operating SystemLinux - Introductions to Linux Operating System
Linux - Introductions to Linux Operating System
 
Getting Started With Linux Administration
Getting Started With Linux AdministrationGetting Started With Linux Administration
Getting Started With Linux Administration
 
Kali linux
Kali linux Kali linux
Kali linux
 
Kali Linux
Kali LinuxKali Linux
Kali Linux
 
Intoduction to VirtualBox English
Intoduction to VirtualBox EnglishIntoduction to VirtualBox English
Intoduction to VirtualBox English
 
Kali Linux
Kali LinuxKali Linux
Kali Linux
 
Redesigning Xen Memory Sharing (Grant) Mechanism
Redesigning Xen Memory Sharing (Grant) MechanismRedesigning Xen Memory Sharing (Grant) Mechanism
Redesigning Xen Memory Sharing (Grant) Mechanism
 

Similar to Clear Linux OS - Architecture Overview

Performance out of the box developers
Performance   out of the box developersPerformance   out of the box developers
Performance out of the box developersMichelle Holley
 
Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...
 Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive... Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...
Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...Databricks
 
Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...Michelle Holley
 
Intel Technologies for High Performance Computing
Intel Technologies for High Performance ComputingIntel Technologies for High Performance Computing
Intel Technologies for High Performance ComputingIntel Software Brasil
 
Intel® Select Solutions for the Network
Intel® Select Solutions for the NetworkIntel® Select Solutions for the Network
Intel® Select Solutions for the NetworkLiz Warner
 
Accelerate Ceph performance via SPDK related techniques
Accelerate Ceph performance via SPDK related techniques Accelerate Ceph performance via SPDK related techniques
Accelerate Ceph performance via SPDK related techniques Ceph Community
 
Accelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent MemoryAccelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent MemoryDatabricks
 
Технологии Intel в центрах обработки данных
Технологии Intel в центрах обработки данныхТехнологии Intel в центрах обработки данных
Технологии Intel в центрах обработки данныхFujitsu Russia
 
Процессор Intel Xeon
Процессор Intel Xeon Процессор Intel Xeon
Процессор Intel Xeon Nick Turunov
 
NFF-GO (YANFF) - Yet Another Network Function Framework
NFF-GO (YANFF) - Yet Another Network Function FrameworkNFF-GO (YANFF) - Yet Another Network Function Framework
NFF-GO (YANFF) - Yet Another Network Function FrameworkMichelle Holley
 
Introduction to container networking in K8s - SDN/NFV London meetup
Introduction to container networking in K8s - SDN/NFV  London meetupIntroduction to container networking in K8s - SDN/NFV  London meetup
Introduction to container networking in K8s - SDN/NFV London meetupHaidee McMahon
 
E5 Intel Xeon Processor E5 Family Making the Business Case
E5 Intel Xeon Processor E5 Family Making the Business Case E5 Intel Xeon Processor E5 Family Making the Business Case
E5 Intel Xeon Processor E5 Family Making the Business Case Intel IT Center
 
DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...
DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...
DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...Andrey Kudryavtsev
 
Технологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связиТехнологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связиCisco Russia
 
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...Intel IT Center
 
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
 Best Practice of Compression/Decompression Codes in Apache Spark with Sophia... Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...Databricks
 

Similar to Clear Linux OS - Architecture Overview (20)

Clear Linux OS - Introduction
Clear Linux OS - IntroductionClear Linux OS - Introduction
Clear Linux OS - Introduction
 
Performance out of the box developers
Performance   out of the box developersPerformance   out of the box developers
Performance out of the box developers
 
Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...
 Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive... Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...
Optimizing Apache Spark Throughput Using Intel Optane and Intel Memory Drive...
 
Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...
 
Intel Technologies for High Performance Computing
Intel Technologies for High Performance ComputingIntel Technologies for High Performance Computing
Intel Technologies for High Performance Computing
 
Intel® Select Solutions for the Network
Intel® Select Solutions for the NetworkIntel® Select Solutions for the Network
Intel® Select Solutions for the Network
 
Accelerate Ceph performance via SPDK related techniques
Accelerate Ceph performance via SPDK related techniques Accelerate Ceph performance via SPDK related techniques
Accelerate Ceph performance via SPDK related techniques
 
Clear Linux Overview and Engagement
Clear Linux Overview and EngagementClear Linux Overview and Engagement
Clear Linux Overview and Engagement
 
Accelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent MemoryAccelerate Your Apache Spark with Intel Optane DC Persistent Memory
Accelerate Your Apache Spark with Intel Optane DC Persistent Memory
 
Технологии Intel в центрах обработки данных
Технологии Intel в центрах обработки данныхТехнологии Intel в центрах обработки данных
Технологии Intel в центрах обработки данных
 
Процессор Intel Xeon
Процессор Intel Xeon Процессор Intel Xeon
Процессор Intel Xeon
 
NFF-GO (YANFF) - Yet Another Network Function Framework
NFF-GO (YANFF) - Yet Another Network Function FrameworkNFF-GO (YANFF) - Yet Another Network Function Framework
NFF-GO (YANFF) - Yet Another Network Function Framework
 
Introduction to container networking in K8s - SDN/NFV London meetup
Introduction to container networking in K8s - SDN/NFV  London meetupIntroduction to container networking in K8s - SDN/NFV  London meetup
Introduction to container networking in K8s - SDN/NFV London meetup
 
E5 Intel Xeon Processor E5 Family Making the Business Case
E5 Intel Xeon Processor E5 Family Making the Business Case E5 Intel Xeon Processor E5 Family Making the Business Case
E5 Intel Xeon Processor E5 Family Making the Business Case
 
DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...
DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...
DUG'20: 11 - Platform Performance Evolution from bring-up to reaching link sa...
 
Технологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связиТехнологии Intel для виртуализации сетей операторов связи
Технологии Intel для виртуализации сетей операторов связи
 
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...
Intel® Xeon® Processor E5-2600 v3 Product Family Application Showcase - Data ...
 
9. intel prez sesiune hw
9. intel prez sesiune hw9. intel prez sesiune hw
9. intel prez sesiune hw
 
Intel on hw
Intel on hwIntel on hw
Intel on hw
 
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
 Best Practice of Compression/Decompression Codes in Apache Spark with Sophia... Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
Best Practice of Compression/Decompression Codes in Apache Spark with Sophia...
 

Recently uploaded

AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024Brian Pichman
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)IES VE
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTopCSSGallery
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024Brian Pichman
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInThousandEyes
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud DataEric D. Schabell
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechProduct School
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsDianaGray10
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfInfopole1
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxKaustubhBhavsar6
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxNeo4j
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptxHansamali Gamage
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 

Recently uploaded (20)

AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development Companies
 
CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024CyberSecurity - Computers In Libraries 2024
CyberSecurity - Computers In Libraries 2024
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
 
Automation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projectsAutomation Ops Series: Session 2 - Governance for UiPath projects
Automation Ops Series: Session 2 - Governance for UiPath projects
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
Extra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdfExtra-120324-Visite-Entreprise-icare.pdf
Extra-120324-Visite-Entreprise-icare.pdf
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptx
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
 
.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx.NET 8 ChatBot with Azure OpenAI Services.pptx
.NET 8 ChatBot with Azure OpenAI Services.pptx
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 

Clear Linux OS - Architecture Overview

  • 1. Clear Linux* OS Architecture Overview Patrick McCarty - Software Engineer *Other names and brands may be claimed as the property of others
  • 2. © 2018 Intel Corporation. Intel, the Intel logo, Intel Inside, the Intel Inside logo, Intel Experience What’s Inside, The Intel Experience What’s Inside logo, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. Intel processors of the same SKU may vary in frequency or power as a result of natural variability in the production process. For more complete information about performance and benchmark results, visit www.intel.com/benchmarks. The cost reduction scenarios described are intended to enable you to get a better understanding of how the purchase of a given Intel based product, combined with a number of situation-specific variables, might affect future costs and savings. Circumstances will vary and there may be unaccounted-for costs related to the use and deployment of a given product. Nothing in this document should be interpreted as either a promise of or contract for a given level of costs or cost reduction. Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate. Optimization Notice: Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor- dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice Revision #20110804. No computer system can be absolutely secure. Intel® Advanced Vector Extensions (Intel® AVX)* provides higher throughput to certain processor operations. Due to varying processor power characteristics, utilizing AVX instructions may cause a) some parts to operate at less than the rated frequency and b) some parts with Intel® Turbo Boost Technology 2.0 to not achieve any or maximum turbo frequencies. Performance varies depending on hardware, software, and system configuration and you can learn more at http://www.intel.com/go/turbo. Available on select Intel® processors. Requires an Intel® HT Technology-enabled system. Your performance varies depending on the specific hardware and software you use. Learn more by visiting http://www.intel.com/info/hyperthreading. § Configurations: The testing was done on Based on fourth-generation Intel Xeon E5-2699 v4 @2.20 GHz processor with 22 cores, 55 MB LLC and 62 GB memory 16 1G hugepages. The testing was conducted in OPNFV Pharos testbed on Pod 12 by VSPERF community engineers Intel, the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. © Intel Corporation Legal Disclaimer
  • 3. Agenda ● Clear Linux* OS Overview ● Performance optimizations ● Use-case focused bundles ● Stateless OS design ● Telemetry ● Updates *Other names and brands may be claimed as the property of others
  • 4. Clear Linux* OS Overview ● Optimized for IA ● Rolling release distribution ● Average of 9 releases per week ● Developer-focused *Other names and brands may be claimed as the property of others
  • 5. Performance Optimizations ● Optimize the entire stack ● Compiler flags ○ Westmere baseline ○ Haswell tuned ● Optimized libraries selected at runtime based on available CPU features ● Performance patches to packages ● Example optimized package: https://github.com/clearlinux- pkgs/opencv/blob/master/opencv.spec Program using OpenCV* AVX2-enabled CPU dynamic linker libopencv_*.so (base) libopencv_*.so (avx2) *Other names and brands may be claimed as the property of others
  • 6. Use-Case Focused Bundles ● Bundles provide use-case driven functionality to end user ● Dependencies resolved at build time on server, not at install or runtime ● Similar to package groups in other distros ● Vertically vs horizontally integrated os-core os-core-update network-basic webserver openssl python-basic application-server kvm-host ansible iproute2 virt-manager scm-server cloud-control *Other names and brands may be claimed as the property of others
  • 7. TRADITIONAL OS User Data System Configuration Operating System CLEAR LINUX* OS User Data System Configuration Operating System Stateless ● OS provides functional and secure default configuration in /usr ● Defaults can be overridden or modified in /etc and the home directory ● Wiping /etc and /var performs a "factory reset", restoring OS default configs *Other names and brands may be claimed as the property of others
  • 8. Stateless – example ● Default telemetrics.conf from operating system in /usr record_expiry=1200 spool_max_size=5120 spool_process_time=900 rate_limit_enabled=true record_burst_limit=1000 record_window_length=15
  • 9. Stateless – example ● Default telemetrics.conf from operating system in /usr ● Custom configuration in /etc record_expiry=1200 spool_max_size=5120 spool_process_time=900 rate_limit_enabled=true record_burst_limit=1000 record_window_length=15 record_expiry=1200 spool_max_size=5120 spool_process_time=900 rate_limit_enabled=false record_burst_limit=1000 record_window_length=15
  • 10. Stateless – example ● Default telemetrics.conf from operating system in /usr ● Custom configuration in /etc record_expiry=1200 spool_max_size=5120 spool_process_time=900 rate_limit_enabled=true record_burst_limit=1000 record_window_length=15 record_expiry=1200 spool_max_size=5120 spool_process_time=900 rate_limit_enabled=false record_burst_limit=1000 record_window_length=15
  • 11. Telemetry ● Opt-in telemetry solution ● Lightweight client service ● Client-side probes send records to help debug software anomalies. ● Probes avoid collecting personally identifiable information and records comply with Privacy Policy*. ● Records are analyzed and displayed in a developer-oriented format on the telemetry server. * https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html
  • 13. Updating ● All installed bundles are updated at once ○ Entire system update (one OS version) ○ QA is done on the entire OS release at once ● Proportional updates ● Auto-update on by default
  • 14. Update content created by mixer tool Upstream Sources Bundle definitions Bundle A data Bundle B data Bundle C data Full chroot Update Creator Update Creator Update Artifacts Swupd clients... swupd clients... Mixing
  • 15. Mixing – Update artifacts Manifests MANIFEST 24 # OS tooling/content format version: 21260 # OS Version this manifest describes previous: 21220 # Previous change to this manifest at this OS version filecount: 13624 # Number of files in the manifest timestamp: 1520706949 # Epoch of creation contentsize: 811403622 # Size, in bytes, of this bundle (not accounting for included bundles) includes: os-core # Bundle included by this bundle F... 0437fc1556fdfe08ee8cfa492094e5c11a86b7b793213767d4f5697d9b437b36 21080 /usr/bin/c_hash F... 4fdebd92c2ad33ad063c8de973b4eafa35d800ff70abe75644172ae6d0b81436 21080 /usr/bin/corelist < 13622 more entries > Manifest of Manifests (MoM) M... 39be958b03625d0507222996f167de279bc2edaec9a1ff45a86f3cdfac83ca6a 21080 desktop-autostart M... 3ac656e9bdb43871f5345cf71c866a67a58d3ce0a2a085efb8e703be4dd3d753 21080 desktop-locales M... 1dbd2354eb2cbf47a871a4d70fc5cee0dc0e6df2c940b03ab6d5ac2edbad594d 21080 dhcp-server
  • 16. Manifest.MoM signed, verification cascades through SHA256 hashes hashes verifiedhashes verified Manifest.MoM Manifest.MoM.sig Manifest.os-core Manifest.go-basic Manifest.shells Manifest.editors Manifest.desktop ... /usr /usr/bin/bash /usr/bin/cat /usr/bin/chmod /usr/bin/rm ... /usr/bin/chown /usr/bin/chroot Mixing – Security
  • 17. Mixing – Update artifacts ● Packs ○ Delta-packs (from version x to y, content difference between versions) Binary deltas ○ Zero-packs (from version 0, complete content of bundle) ● Full files (for fallback) ○ Compressed full files available for download if pack download/extraction fails
  • 18. NON-ATOMIC ms-s durationATOMIC Download + verify MoM Download + extract packs Apply delta files and stage Verify pack contents with manifests Rename staged to final Update Artifacts Updating – Client Operation
  • 19. Clear Linux bundles and content Mixer Update Artifacts swupd clients... User bundles and content Creating Custom Mixes Useful for teams that want to provide their own content on top of Clear Linux* OS content for development, testing, etc. *Other names and brands may be claimed as the property of others
  • 20. mixin Useful for individual users that want to add their own content User adds package Mixer Local Artifacts Upstream Artifacts Merge swupd client Side-loading Custom Content
  • 21. Clear Linux* OS ● Rolling release security updates ● Stateless OS design ● Performance focused ● Use-case optimized bundles ● Fast, secure, and reliable updates *Other names and brands may be claimed as the property of others
  • 22. Contact details Patrick McCarty pmccarty on #clearlinux (freenode) More resources: Project site: clearlinux.org Forum: community.clearlinux.org Git repos: github.com/clearlinux github.com/clearlinux-pkgs