Research if we can dynamically reserve memory and CPU for nodes.

 Feature Overview

This will be phase 1 of Internationalization of the OpenShift Console.

 Phase 1 will include the following:

1. UI based language Selector instead of using browser detection
2. Externalize all hard coded strings in the client code including all OpenShift static plugins
   1. Admin Console
   2. Dev Console
   3. Serverless
   4. Pipelines
   5. CNV
   6. OCS
   7. CSO
3. Localized Date\Time
4. Setup all processes, infrastructure, and testing required
5. We will start with support for Chinese and Japanese lang

Phase 1 will not include:

1. Dynamically generated UI (Operator, OpenAPIV3Schema)
   1. Operators that surface informational messages may not have translations available
2. Strings from non client code
   1. This may include items such as events surfaced from Kuberenetes, alerts, and error messages displayed to the user or in logs
3. Localization of logging messages at any level is not in scope
4. Any CLI
5. Language support for left to right languages ie Arabic

Initial List of Languages to Support

---------- 4.7* ----------

1. Japanese - Code: ja 
2. Chinese - Code: zh_CN, zh_TW 
3. Korean - Code: ko

*This will be based on the ability to get all the strings externalized, there is a good chance this gets pushed to 4.8.

---------- Post 4.7 ----------

1. Spanish: - Code: es_419, es 
2. German: - Code: de
3. French - Code: fr
4. Italian - Code: it
5. Portuguese - Code: pt_BR
6. Korean - Code: ko
7. Hindi - Code: hi

POC

 Initial POC PR

Goals

Internationalization has become table stakes. OpenShift Console needs to support different languages in each of the major markets. This is key functionality that will help unlock sales in different regions.

Requirements

Out of Scope

1. Dynamically generated UI (Operator, OpenAPIV3Schema)
   1. Operators that surface informational messages may not have translations available
2. Strings from non client code
   1. This may include items such as events surfaced from Kuberenetes, alerts, and error messages displayed to the user or in logs
3. Localization of logging messages at any level is not in scope
4. Any CLI support
5. Language support for left to right languages ie. Arabic

Assumptions

• Each static plugin team will be responsible for externalizing all their client code strings.
• Quick Starts will need to be translated.

Customer Considerations

We are rolling this feature in phases, based on customer feedback, there may be no phase 2.

Documentation Considerations

I believe documentation already supports a large language set.

Feature Overview

Openshift Sandboxed Containers provide the ability to add an additional layer of isolation through virtualization for many workloads. The main way to enable the use of katacontainers on an Openshift Cluster is by first installing the Operator (for more information about operator enablement check [1]).

Once the feature is enabled on the cluster, it just a matter of a one-liner YAML modification on the pod/deployment level to run the workload using katacontianers. That might sound easy for some, but for others who don't care about YAML they might want more abstractions on how to use katacontainers for their workloads.

This feature covers all the efforts required to integrate and present Kata in Openshift UI (console) to cater to all user personas.

Background, and strategic fit

To enable for users to adopt Kata as a runtime, it is important to make it easy to use. Adding hook-points in the UI with ease-of-use as a goal in mind is one way to bring in more users.

Goal(s)

The main goal of this feature is to make sure that:

1. It is easy for users to find out how to use/enable Openshift Sandboxed Containers on their clusters (e.g., Getting started guide in the UI).
2. Cluster-admins are able to differentiate between normal pods and Kata pods.
3. Developers (application, CNF, ...) have an easy way to create Katacontiners (without peeking at YAMLs)
4. Application End-users are able to collectively activate kata on their app packages/content (e.g., Helm, odo,...)

Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?
• Does this feature have doc impact?
• New Content, Updates to existing content, Release Note, or No Doc Impact
• If unsure and no Technical Writer is available, please contact Content Strategy.
• What concepts do customers need to understand to be successful in [action]?
• How do we expect customers will use the feature? For what purpose(s)?
• What reference material might a customer want/need to complete [action]?
• Is there source material that can be used as a reference for the Technical Writer in writing the content? If yes, please link if available.
• What is the doc impact (New Content, Updates to existing content, or Release Note)?

References

[1] https://issues.redhat.com/browse/KATA-429?jql=project %3D KATA AND issuetype %3D Feature

Why?

• Decouple control and data plane. 
  • Customers do not pay Red Hat more to run HyperShift control planes and supporting infrastructure than Standalone control planes and supporting infrastructure.
• Improve security
  • Shift credentials out of cluster that support the operation of core platform vs workload
• Improve cost
  • Allow a user to toggle what they don’t need.
  • Ensure a smooth path to scale to 0 workers and upgrade with 0 workers.

Assumption

• A customer will be able to associate a cluster as “Infrastructure only”
• E.g. one option: management cluster has role=master, and role=infra nodes only, control planes are packed on role=infra nodes
• OR the entire cluster is labeled infrastructure , and node roles are ignored.

• Anything that runs on a master node by default in Standalone that is present in HyperShift MUST be hosted and not run on a customer worker node.

Doc: https://docs.google.com/document/d/1sXCaRt3PE0iFmq7ei0Yb1svqzY9bygR5IprjgioRkjc/edit 

Review the OVN Interconnect proposal, figure out the work that needs to be done in ovn-kubernetes to be able to move to this new OVN architecture. 

Why is this important?

OVN IC will be the model used in Hypershift. 

Feature Overview

RHEL CoreOS should be updated to RHEL 9.2 sources to take advantage of newer features, hardware support, and performance improvements.

Requirements

• RHEL 9.x sources for RHCOS builds starting with OCP 4.13 and RHEL 9.2.

(Optional) Use Cases

• 9.2 Preview via Layering No longer necessary assuming we stay the course of going all in on 9.2

Assumptions

• ...

Customer Considerations

• ...

Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?
• Does this feature have doc impact?
• New Content, Updates to existing content, Release Note, or No Doc Impact
• If unsure and no Technical Writer is available, please contact Content Strategy.
• What concepts do customers need to understand to be successful in [action]?
• How do we expect customers will use the feature? For what purpose(s)?
• What reference material might a customer want/need to complete [action]?
• Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available.
• What is the doc impact (New Content, Updates to existing content, or Release Note)?

Feature Overview

• Kubernetes offers different ways to consume, one could request persistent volumes that survive pod termination or ask for a ephemeral storage space that will be consumed during the lifetime of the pod.
• This feature tracks the improvements around ephemeral storage as some workloads rely on reliable temporary storage space such as batch jobs, caching services or any app that does not care whether the data is stored persistently across restarts

Goals

As described in the kubernetes "ephemeral volumes" documentation this features tracks GA and improvements in

• emptyDir: empty at Pod startup, with storage coming locally from the kubelet base directory (usually the root disk) or RAM
• CSI ephemeral volumes: similar to the previous volume kinds, but provided by special CSI drivers which specifically support this feature
• generic ephemeral volumes, which can be provided by all storage drivers that also support persistent volumes

OCPPLAN-9193 Implemented local ephemeral capacity management as well as CSI Generic ephemeral volume. This feature tracks the remaining work to GA CSI ephemeral in-inline volume, specially the admission plugin to make the feature secure and prevent any insecure driver from using it. Ephemeral in-line is required by some CSI as key feature to operate (e.g SecretStore CSI), ODF is also planning to GA ephemeral in-line with ceph CSI. 

Requirements

Use Cases

This Section:

• As an OCP user I want to consume ephemeral storage for my workload
• As an OCP user I would like to include my PV definition directly in my app definition
• As an OCP admin I would like to offer ephemeral volumes to my users though CSI
• As a partner I would like to onboard a driver that relies on CSI inline volumes

Customer Considerations

• Make sure each ephemeral volume option is clearly identified and documented for each purpose.

• Make sure we highlight ephemeral volume options that require a specific driver support

Note: Replace text in red with details of your feature request.

Feature Overview

Extend the Workload Partitioning feature to support multi-node clusters.

Goals

Customers running RAN workloads on C-RAN Hubs (i.e. multi-node clusters) that want to maximize the cores available to the workloads (DU) should be able to utilize WP to isolate CP processes to reserved cores.

Requirements

A list of specific needs or objectives that a Feature must deliver to satisfy the Feature. Some requirements will be flagged as MVP. If an MVP gets shifted, the feature shifts.  If a non MVP requirement slips, it does not shift the feature.

Describe Use Cases (if needed)

< How will the user interact with this feature? >

< Which users will use this and when will they use it? >

< Is this feature used as part of current user interface? >

Out of Scope

Background, and strategic fit

< What does the person writing code, testing, documenting need to know? >

Assumptions

< Are there assumptions being made regarding prerequisites and dependencies?>

< Are there assumptions about hardware, software or people resources?>

Customer Considerations

< Are there specific customer environments that need to be considered (such as working with existing h/w and software)?>

< Are there Upgrade considerations that customers need to account for or that the feature should address on behalf of the customer?>

<Does the Feature introduce data that could be gathered and used for Insights purposes?>

Documentation Considerations

< What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)? >

< What does success look like?>

< Does this feature have doc impact?  Possible values are: New Content, Updates to existing content,  Release Note, or No Doc Impact>

< If unsure and no Technical Writer is available, please contact Content Strategy. If yes, complete the following.>

• <What concepts do customers need to understand to be successful in [action]?>

• <How do we expect customers will use the feature? For what purpose(s)?>

• <What reference material might a customer want/need to complete [action]?>

• <Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available. >

• <What is the doc impact (New Content, Updates to existing content, or Release Note)?>

Interoperability Considerations

< Which other products and versions in our portfolio does this feature impact?>

< What interoperability test scenarios should be factored by the layered product(s)?>

Questions

The goal of this effort is to leverage OVN Kubernetes SDN to satisfy networking requirements of both traditional and modern virtualization. This Feature describes the envisioned outcome and tracks its implementation.

Current state

In its current state, OpenShift Virtualization provides a flexible toolset allowing customers to connect VMs to the physical network. It also has limited secondary overlay network capabilities and Pod network support.

It suffers from several gaps: Topology of the default pod network is not suitable for typical VM workload - due to that we are missing out on many of the advanced capabilities of OpenShift networking, and we also don't have a good solution for public cloud. Another problem is that while we provide plenty of tools to build a network solution, we are not very good in guiding cluster administrators configuring their network, making them rely on their account team.

Desired outcome

Provide:

• Networking solution for public cloud
• Advanced SDN networking functionality such as IPAM, routed ingress, DNS and cloud-native integration
• Ability to host traditional VM workload imported from other virtualization platforms

... while maintaining networking expectations of a typical VM workload:

• Sticky IPs allowing seamless live migration
• External IP reflected inside the guest, i.e. no NAT for east-west traffic

Additionally, make our networking configuration more accessible to newcomers by providing a finite list of user stories mapped to recommended solutions.

Timeline

Complete milestones

• CNV-17002 Flat overlay (v4.14)
• CNV-24261 Localnet (v4.15)
• CNV-24259 ipBlock policies (v4.15)

Next actions (tentative)

4.17 user defined networks TP:

• CNV-24260 IPAM
• SDN-4919 Contributing to OVN's work
• CNV-35223 Self-service

4.17 user defined networks DP:

• CNV-42490 Networks binding (integration between user-defined networks and KubeVirt)
• CNV-33753 Egress

4.17 other work:

• CNV-25451 *Selector policies

4.18 user-defined networks for public cloud GA:

• CNV-42637 Live-migration (not to be confused with seamless live migration CNV-27147)
• CNV-41302 IPAM (graduation on both CNV and OCP)
• CNV-44233 Network binding (graduation, removing feature gates and deploying it with CNV)
• CNV-44224 Egress (graduation, just testing)
• CNV-24258 LoadBalancer ingress (straight to GA unless we see there are integration points with CNV)

4.18 user-defined networks, other, GA:

• CNV-39842 Self-service (graduation)

4.18 localnet enhancements:

• CNV-38180 MAC spoof filtering
• CNV-44214 Promiscuous mode
• OCPBUGS-31679 (needs to be reopened) Single OVN mapping per bridge

User stories

You can find more info about this effort in https://docs.google.com/document/d/1jNr0E0YMIHsHu-aJ4uB2YjNY00L9TpzZJNWf3LxRsKY/edit

• Feature complete including (next line)
• OVN-K

Market Problem

As a stakeholder aiming to adopt KubeSaw as a Namespace-as-a-Service solution, I want the project to provide streamlined tooling and a clear code-base, ensuring seamless adoption and integration into my clusters.

Why it Matters

Efficient adoption of KubeSaw, especially as a Namespace-as-a-Service solution, relies on intuitive tooling and a transparent codebase. Improving these aspects will empower stakeholders to effortlessly integrate KubeSaw into their Kubernetes clusters, ensuring a smooth transition to enhanced namespace management.

Illustrative User Stories

As a Stakeholder, I want streamlined setup of the KubeSaw project and fully automated way of upgrading this setup aling with the updates of the installation.

Expected Outcomes

• Intuitive and user-friendly tooling for seamless configuration and management of KubeSaw instance.
• A transparent and well-documented codebase, facilitating a quick understanding of KubeSaw internals.

Effect

The expected outcome within the market is both growth and retention. The improved tooling and codebase will attract new stakeholders (growth) and enhance the experience for existing users (retention) by providing a straightforward path to adopting KubeSaw's Namespace-as-a-Service features in their clusters.

Partner

• Developer Sandbox
• Konflux

Additional/tangential areas for future development

• Integration with popular Kubernetes management platforms and tooling for enhanced interoperability.
• Regular updates to compatibility matrices to support evolving Kubernetes technologies.
• Collaboration with stakeholders to gather feedback and continuously improve the integration experience, including advanced namespace management features tailored to user needs.

Feature Overview

We drive OpenShift cross-market customer success and new customer adoption with constant improvements and feature additions to the existing capabilities of our OpenShift Core Networking (SDN and Network Edge). This feature captures that natural progression of the product.

Goals

• Feature enhancements (performance, scale, configuration, UX, ...)
• Modernization (incorporation and productization of new technologies)

Requirements

• Core Networking Stability
• Core Networking Performance and Scale
• Core Neworking Extensibility (Multus CNIs)
• Core Networking UX (Observability)
• Core Networking Security and Compliance

In Scope

• Network Edge (ingress, DNS, LB)
• SDN (CNI plugins, openshift-sdn, OVN, network policy, egressIP, egress Router, ...)
• Networking Observability

Out of Scope

There are definitely grey areas, but in general:

• CNV
• Service Mesh
• CNF

Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?
• Does this feature have doc impact?
• New Content, Updates to existing content, Release Note, or No Doc Impact
• If unsure and no Technical Writer is available, please contact Content Strategy.
• What concepts do customers need to understand to be successful in [action]?
• How do we expect customers will use the feature? For what purpose(s)?
• What reference material might a customer want/need to complete [action]?
• Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available.
• What is the doc impact (New Content, Updates to existing content, or Release Note)?

Feature Overview

Plugin teams need a mechanism to extend the OCP console that is decoupled enough so they can deliver at the cadence of their projects and not be forced in to the OCP Console release timelines.

The OCP Console Dynamic Plugin Framework will enable all our plugin teams to do the following:

• Extend the Console
• Deliver UI code with their Operator
• Work in their own git Repo
• Deliver at their own cadence

Goals

• • Operators can deliver console plugins separate from the console image and update plugins when the operator updates.
  • The dynamic plugin API is similar to the static plugin API to ease migration.
  • Plugins can use shared console components such as list and details page components.
  • Shared components from core will be part of a well-defined plugin API.
  • Plugins can use Patternfly 4 components.
  • Cluster admins control what plugins are enabled.
  • Misbehaving plugins should not break console.
  • Existing static plugins are not affected and will continue to work as expected.

Out of Scope

• • Initially we don't plan to make this a public API. The target use is for Red Hat operators. We might reevaluate later when dynamic plugins are more mature.
  • We can't avoid breaking changes in console dependencies such as Patternfly even if we don't break the console plugin API itself. We'll need a way for plugins to declare compatibility.
  • Plugins won't be sandboxed. They will have full JavaScript access to the DOM and network. Plugins won't be enabled by default, however. A cluster admin will need to enable the plugin.
  • This proposal does not cover allowing plugins to contribute backend console endpoints.

Requirements

 
 Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?

• Does this feature have doc impact?  

• New Content, Updates to existing content,  Release Note, or No Doc Impact

• If unsure and no Technical Writer is available, please contact Content Strategy.

• What concepts do customers need to understand to be successful in [action]?

• How do we expect customers will use the feature? For what purpose(s)?

• What reference material might a customer want/need to complete [action]?

• Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available.

• What is the doc impact (New Content, Updates to existing content, or Release Note)?

Feature Overview

• This Section:* High-Level description of the feature ie: Executive Summary

• Note: A Feature is a capability or a well defined set of functionality that delivers business value. Features can include additions or changes to existing functionality. Features can easily span multiple teams, and multiple releases.

Goals

• This Section:* Provide high-level goal statement, providing user context and expected user outcome(s) for this feature

• …

Requirements

• This Section:* A list of specific needs or objectives that a Feature must deliver to satisfy the Feature.. Some requirements will be flagged as MVP. If an MVP gets shifted, the feature shifts. If a non MVP requirement slips, it does not shift the feature.

(Optional) Use Cases

This Section: 

• Main success scenarios - high-level user stories

• Alternate flow/scenarios - high-level user stories

• ...

Questions to answer…

• ...

Out of Scope

• …

Background, and strategic fit

This Section: What does the person writing code, testing, documenting need to know? What context can be provided to frame this feature.

Assumptions

• ...

Customer Considerations

• ...

Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?

• Does this feature have doc impact?  

• New Content, Updates to existing content,  Release Note, or No Doc Impact

• If unsure and no Technical Writer is available, please contact Content Strategy.

• What concepts do customers need to understand to be successful in [action]?

• How do we expect customers will use the feature? For what purpose(s)?

• What reference material might a customer want/need to complete [action]?

• Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available.

• What is the doc impact (New Content, Updates to existing content, or Release Note)?

Feature Overview (aka. Goal Summary)

As a result of Hashicorp's license change to BSL, Red Hat OpenShift needs to remove the use of Hashicorp's Terraform from the installer – specifically for OpenStack deployments which currently use Terraform for setting up the infrastructure.

To avoid an increased support overhead once the license changes at the end of the year, we want to provision OpenStack infrastructure without the use of Terraform.

Requirements (aka. Acceptance Criteria):

• The OpenStack Installer no longer contains or uses Terraform.
• The new provider should aim to provide the same results and have parity with the existing OpenStack Terraform provider. Specifically, we should aim for feature parity against the install config and the cluster it creates to minimize impact on existing customers' UX.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios. Initial completion during Refinement status.

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin. Initial completion during Refinement status.

Out of Scope

High-level list of items that are out of scope. Initial completion during Refinement status.

Background

Provide any additional context is needed to frame the feature. Initial completion during Refinement status.

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature. Initial completion during Refinement status.

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs. If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact? What interoperability test scenarios should be factored by the layered products? Initial completion during Refinement status.

Feature Overview (aka. Goal Summary)

As a result of Hashicorp's license change to BSL, Red Hat OpenShift needs to remove the use of Hashicorp's Terraform from the installer – specifically for IPI deployments which currently use Terraform for setting up the infrastructure.

To avoid an increased support overhead once the license changes at the end of the year, we want to provision OpenShift on the existing supported providers' infrastructure without the use of Terraform.

This feature will be used to track all the CAPI preparation work that is common for all the supported providers

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios. Initial completion during Refinement status.

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin. Initial completion during Refinement status.

Out of Scope

High-level list of items that are out of scope. Initial completion during Refinement status.

Background

Provide any additional context is needed to frame the feature. Initial completion during Refinement status.

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature. Initial completion during Refinement status.

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs. If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact? What interoperability test scenarios should be factored by the layered products? Initial completion during Refinement status.

Epic Goal*

Provide a long term solution to SELinux context labeling in OCP.

 
Why is this important? (mandatory)

As of today when selinux is enabled, the PV's files are relabeled when attaching the PV to the pod, this can cause timeout when the PVs contains lot of files as well as overloading the storage backend.

https://access.redhat.com/solutions/6221251 provides few workarounds until the proper fix is implemented. Unfortunately these workaround are not perfect and we need a long term seamless optimised solution.

This feature tracks the long term solution where the PV FS will be mounted with the right selinux context thus avoiding to relabel every file.

 
Scenarios (mandatory) 

Provide details for user scenarios including actions to be performed, platform specifications, and user personas.  

1. Apply new context when there is none
2. Change context of all files/folders when changing context
3. RWO & RWX PVs
   1. ReadWriteOncePod PVs first
   2. RWX PV in a second phase

As we are relying on mount context there should not be any relabeling (chcon) because all files / folders will inherit the context from the mount context

More on design & scenarios in the KEP  and related epic STOR-1173

Dependencies (internal and external) (mandatory)

None for the core feature

However the driver will have to set SELinuxMountSupported to true in the CSIDriverSpec to enable this feature. 

Contributing Teams(and contacts) (mandatory) 

Our expectation is that teams would modify the list below to fit the epic. Some epics may not need all the default groups but what is included here should accurately reflect who will be involved in delivering the epic.

• Development - STOR
• Documentation - STOR
• QE - STOR
• PX - 
• Others -

Done - Checklist (mandatory)

The following points apply to all epics and are what the OpenShift team believes are the minimum set of criteria that epics should meet for us to consider them potentially shippable. We request that epic owners modify this list to reflect the work to be completed in order to produce something that is potentially shippable.

• CI Testing -  Basic e2e automationTests are merged and completing successfully
• Documentation - Content development is complete.
• QE - Test scenarios are written and executed successfully.
• Technical Enablement - Slides are complete (if requested by PLM)
• Engineering Stories Merged
• All associated work items with the Epic are closed
• Epic status should be “Release Pending” 

Goal of this feature is to add support to 

• telemetry
• nmstate ipv6
• nmstate net2net

Why is this important?

• without API, customers are forced to use MCO. this brings with it a set of limitations (mainly reboot per change and the fact that config is shared among each pool, can't do per node configuration)
• better upgrade solution will give us the ability to support a single host based implementation
• telemetry will give us more info on how widely is ipsec used.

Acceptance Criteria

• CI - MUST be running successfully with tests automated
• Release Technical Enablement - Provide necessary release enablement details and documents.
• Must allow for the possibility of offloading the IPsec encryption to a SmartNIC.

• nmstate
• k8s-nmstate
• easier mechanism for cert injection (??)
• telemetry
• improve ci and test coverage
   

Dependencies (internal and external)

1.  nmstate tasks

Related:

• ITUP-44 - OpenShift support for North-South OVN IPSec
• HATSTRAT-33 - Encrypt All Traffic to/from Cluster (aka IPSec as a Service)

Previous Work (Optional):

1. SDN-717 - Support IPSEC on ovn-kubernetes
2. SDN-3604 - Fully supported non-GA N-S IPSec implementation using machine config.

Open questions::

1. …

Done Checklist

• CI - CI is running, tests are automated and merged.
• Release Enablement <link to Feature Enablement Presentation>
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub Issue>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub Issue>
• DEV - Downstream build attached to advisory: <link to errata>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Downstream documentation merged: <link to meaningful PR>

Feature Overview (aka. Goal Summary)  

Support network isolation and multiple primary networks (with the possibility of overlapping IP subnets) without having to use Kubernetes Network Policies.

Goals (aka. expected user outcomes)

• Provide a configurable way to indicate that a pod should be connected to a unique network of a specific type via its primary interface.
• Allow networks to have overlapping IP address space.
• The primary network defined today will remain in place as the default network that pods attach to when no unique network is specified.
• Support cluster ingress/egress traffic for unique networks, including secondary networks.
• Support for ingress/egress features where possible, such as:
  • EgressQoS
  • EgressService
  • EgressIP
  • Load Balancer Services

Requirements (aka. Acceptance Criteria):

• Support for 10,000 namespaces
•  

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Design Document

Use Cases (Optional):

• As an OpenStack or vSphere/vCenter user, who is migrating to OpenShift Kubernetes, I want to guarantee my OpenStack/vSphere tenant network isolation remains intact as I move into Kubernetes namespaces.
• As an OpenShift Kubernetes user, I do not want to have to rely on Kubernetes Network Policy and prefer to have native network isolation per tenant using a layer 2 domain.
• As an OpenShift Network Administrator with multiple identical application deployments across my cluster, I require a consistent IP-addressing subnet per deployment type. Multiple applications in different namespaces must always be accessible using the same, predictable IP address.

Questions to Answer (Optional):

Out of Scope

• Multiple External Gateway (MEG) Support - support will remain for default primary network.
• Pod Ingress support - support will remain for default primary network.
• Cluster IP Service reachability across networks. Services and endpoints will be available only within the unique network.
• Allowing different service CIDRs to be used in different networks.
• Localnet will not be supported initially for primary networks.
• Allowing multiple primary networks per namespace.
• Allow connection of multiple networks via explicit router configuration. This may be handled in a future enhancement.
• Hybrid overlay support on unique networks.

Background

OVN-Kubernetes today allows multiple different types of networks per secondary network: layer 2, layer 3, or localnet. Pods can be connected to different networks without discretion. For the primary network, OVN-Kubernetes only supports all pods connecting to the same layer 3 virtual topology.

As users migrate from OpenStack to Kubernetes, there is a need to provide network parity for those users. In OpenStack, each tenant (analog to a Kubernetes namespace) by default has a layer 2 network, which is isolated from any other tenant. Connectivity to other networks must be specified explicitly as network configuration via a Neutron router. In Kubernetes the paradigm is the opposite; by default all pods can reach other pods, and security is provided by implementing Network Policy.

Network Policy has its issues:

• it can be cumbersome to configure and manage for a large cluster
• it can be limiting as it only matches TCP, UDP, and SCTP traffic
• large amounts of network policy can cause performance issues in CNIs

With all these factors considered, there is a clear need to address network security in a native fashion, by using networks per user to isolate traffic instead of using Kubernetes Network Policy.

Therefore, the scope of this effort is to bring the same flexibility of the secondary network to the primary network and allow pods to connect to different types of networks that are independent of networks that other pods may connect to.

Customer Considerations

Documentation Considerations

Interoperability Considerations

Test scenarios:

• E2E upstream and downstream jobs covering supported features across multiple networks.
• E2E tests ensuring network isolation between OVN networked and host networked pods, services, etc.
• E2E tests covering network subnet overlap and reachability to external networks.
• Scale testing to determine limits and impact of multiple unique networks.

Elaborate more dashboards (monitoring dashboards, accessible from menu Observe > Dashboards ; admin perspective) related to networking.

Start with just a couple of areas:

• Host network dashboard (using node-exporter network / netstat metrics - related to CMO)
• OVN/OVS health dashboard (using ovn/ovs metrics)
• Ingress dashboard (routes, shards stats) related to Ingress operator / netedge team
  (- DNS dashboard, if time)

More info/discussion in this work doc: https://docs.google.com/document/d/1ByNIJiOzd6w5csFYpC27NdOydnBg8Tx45uL4-7v-aCM/edit

Feature Overview (aka. Goal Summary)  

Customers have requested the ability to have the ability to apply tolerations to the HCP control plane pods. This provides the flexibility to have the HCP pods scheduled to nodes with taints applied to them that are not currently tolerated by default.

API 

Add new field to HostedCluster. hc.Spec.Tolerations

Tolerations   []corev1.Toleration `json:"tolerations,omitempty"` 

Implementation

In support/config/deployment.go, add hc.spec.tolerations from hc when generating the default config. This will cause the toleration to naturally get spread to the deployments and statefulsets.

CLI

Add new cli argument called –tolerations to the hcp cli tool during cluster creation. This argument should be able to be set multiple times. The syntax of the field should follow the convention set by the kubectl client tool when setting a taint on a node.

For example, the kubectl client tool can be used to set the following taint on a node.

kubectl taint nodes node1 key1=value1:NoSchedule

And then the hcp cli tool should be able to add a toleration for this taint during creation with the following cli arg.

hcp cluster create kubevirt –toleration “key1=value1:noSchedule” …

Goals (aka. expected user outcomes)

• Support for customer defined tolerations for HCP pods

  Requirements (aka. Acceptance Criteria):

A list of specific needs or objectives that a feature must deliver in order to be considered complete.  Be sure to include nonfunctional requirements such as security, reliability, performance, maintainability, scalability, usability, etc.  Initial completion during Refinement status.

<enter general Feature acceptance here>

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

<your text here>

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

<your text here>

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

<your text here>

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

<your text here>

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>

Continue scale testing and performance improvements for ovn-kubernetes

Feature Overview (aka. Goal Summary)  

Enable CPU manager on s390x.

Why is this important?

CPU manager is an important component to manage performance of OpenShift and utilize the respective platforms.

Goals (aka. expected user outcomes)

Enable CPU manager on s390x.

Requirements (aka. Acceptance Criteria):

CPU manager works on s390x.

Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

Use Cases (Optional):

Include use case diagrams, main success scenarios, alternative flow scenarios.  Initial completion during Refinement status.

<your text here>

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

<your text here>

Out of Scope

High-level list of items that are out of scope.  Initial completion during Refinement status.

<your text here>

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

<your text here>

Customer Considerations

Provide any additional customer-specific considerations that must be made when designing and delivering the Feature.  Initial completion during Refinement status.

<your text here>

Documentation Considerations

Provide information that needs to be considered and planned so that documentation will meet customer needs.  If the feature extends existing functionality, provide a link to its current documentation. Initial completion during Refinement status.

<your text here>

Interoperability Considerations

Which other projects, including ROSA/OSD/ARO, and versions in our portfolio does this feature impact?  What interoperability test scenarios should be factored by the layered products?  Initial completion during Refinement status.

<your text here>

Feature Overview (aka. Goal Summary)  

OVN Kubernetes Developer's Preview for BGP as a routing protocol for User Defined Network (Segmentation) pod and VM addressability via common data center networking removing the need to negotiate NAT at the cluster's edge.

Goals (aka. expected user outcomes)

OVN-Kubernetes currently has no native routing protocol integration, and relies on a Geneve overlay for east/west traffic, as well as third party operators to handle external network integration into the cluster. The purpose of this Developer's Preview enhancement is to introduce BGP as a supported routing protocol with OVN-Kubernetes. The extent of this support will allow OVN-Kubernetes to integrate into different BGP user environments, enabling it to dynamically expose cluster scoped network entities into a provider’s network, as well as program BGP learned routes from the provider’s network into OVN. In a follow-on release, this enhancement will provide support for EVPN, which is a common data center networking fabric that relies on BGP.

Requirements (aka. Acceptance Criteria):

• Provide a user-facing API to allow configuration of iBGP or eBGP peers, along with typical BGP configurations to include communities, route targets, vpnv4/v6, etc
• Support for advertising Egress IP addresses
• Enable BFD to BGP peers
• Support EVPN configuration and integration with a user’s DC fabric, along with MAC-VRFs and IP-VRFs
• ECMP routing support within OVN for BGP learned routes
   
  Anyone reviewing this Feature needs to know which deployment configurations that the Feature will apply to (or not) once it's been completed.  Describe specific needs (or indicate N/A) for each of the following deployment scenarios. For specific configurations that are out-of-scope for a given release, ensure you provide the OCPSTRAT (for the future to be supported configuration) as well.

  Deployment considerations	List applicable specific needs (N/A = not applicable)
  Self-managed, managed, or both
  Classic (standalone cluster)
  Hosted control planes
  Multi node, Compact (three node), or Single node (SNO), or all
  Connected / Restricted Network
  Architectures, e.g. x86_x64, ARM (aarch64), IBM Power (ppc64le), and IBM Z (s390x)
  Operator compatibility
  Backport needed (list applicable versions)
  UI need (e.g. OpenShift Console, dynamic plugin, OCM)
  Other (please specify)

Design Document

Use Cases (Optional):

• Integration with 3rdparty load balancers that send packets directly to OpenShift nodes with the destination IP address of a targeted pod, without needing custom operators to detect which node a pod is scheduled to and then add routes into the load balancer to send the packet to the right node.

Questions to Answer (Optional):

Out of Scope

• EVPN integration
• Support of any other routing protocol
• Running separate BGP instances per VRF network
• Support for any other type of L3VPN with BGP, including MPLS
• Providing any type of API or operator to automatically connect two Kubernetes clusters via L3VPN
• Replacing the support that MetalLB provides today for advertising service IPs
• Asymmetric Integrated Routing and Bridging (IRB) with EVPN

Background

BGP

Importing Routes from the Provider Network
Today in OpenShift there is no API for a user to be able to configure routes into OVN. In order for a user to change how cluster traffic is routed egress into the cluster, the user leverages local gateway mode, which forces egress traffic to hop through the Linux host's networking stack, where a user can configure routes inside of the host via NM State. This manual configuration would need to be performed and maintained across nodes and VRFs within each node.

Additionally, if a user chooses to not manage routes within the host and use local gateway mode, then by default traffic is always sent to the default gateway. The only other way to affect egress routing is by using the Multiple External Gateways (MEG) feature. With this feature the user may choose to have multiple different egress gateways per namespace to send traffic to.

As an alternative, configuring BGP peers and which route-targets to import would eliminate the need to manually configure routes in the host, and would allow dynamic routing updates based on changes in the provider’s network.

Exporting Routes into the Provider Network
There exists a need for provider networks to learn routes directly to services and pods today in Kubernetes. Metal LB is already one solution whereby load balancer IPs are advertised by BGP to provider networks, and this feature development does not intend to duplicate or replace the function of Metal LB. Metal LB should be able to interoperate with OVN-Kubernetes, and be responsible for advertising services to a provider’s network.

However, there is an alternative need to advertise pod IPs on the provider network. One use case is integration with 3rd party load balancers, where they terminate a load balancer and then send packets directly to OCP nodes with the destination IP address being the pod IP itself. Today these load balancers rely on custom operators to detect which node a pod is scheduled to and then add routes into its load balancer to send the packet to the right node.

By integrating BGP and advertising the pod subnets/addresses directly on the provider network, load balancers and other entities on the network would be able to reach the pod IPs directly.

EVPN (to be integrated with BGP in a follow-on release targeting 4.18)

Extending OVN-Kubernetes VRFs into the Provider Network
This is the most powerful motivation for bringing support of EVPN into OVN-Kubernetes. A previous development effort enabled the ability to create a network per namespace (VRF) in OVN-Kubernetes, allowing users to create multiple isolated networks for namespaces of pods. However, the VRFs terminate at node egress, and routes are leaked from the default VRF so that traffic is able to route out of the OCP node. With EVPN, we can now extend the VRFs into the provider network using a VPN. This unlocks the ability to have L3VPNs that extend across the provider networks.

Utilizing the EVPN Fabric as the Overlay for OVN-Kubernetes
In addition to extending VRFs to the outside world for ingress and egress, we can also leverage EVPN to handle extending VRFs into the fabric for east/west traffic. This is useful in EVPN DC deployments where EVPN is already being used in the TOR network, and there is no need to use a Geneve overlay. In this use case, both layer 2 (MAC-VRFs) and layer 3 (IP-VRFs) can be advertised directly to the EVPN fabric. One advantage of doing this is that with Layer 2 networks, broadcast, unknown-unicast and multicast (BUM) traffic is suppressed across the EVPN fabric. Therefore the flooding domain in L2 networks for this type of traffic is limited to the node.

Multi-homing, Link Redundancy, Fast Convergence
Extending the EVPN fabric to OCP nodes brings other added benefits that are not present in OCP natively today. In this design there are at least 2 physical NICs and links leaving the OCP node to the EVPN leaves. This provides link redundancy, and when coupled with BFD and mass withdrawal, it can also provide fast failover. Additionally, the links can be used by the EVPN fabric to utilize ECMP routing.

Customer Considerations

• For customers using MetalLB, it will continue to function correctly regardless of this development.

Documentation Considerations

Interoperability Considerations

• Multiple External Gateways (MEG)
• Egress IP
• Services
• Egress Service
• Egress Firewall
• Egress QoS

Epic Goal

• Review design and development PRs that require feedback from NE team.

Why is this important?

• Customer requires certificates to be managed by cert-manager on configured/newly added routes.

Acceptance Criteria

• All PRs are reviewed and merged.

Dependencies (internal and external)

1. CFE team dependency for addressing review suggestions.

Done Checklist

• DEV - All related PRs are merged.

Goal:
Support enablement of dual-stack VIPs on existing clusters created as dual-stack but at a time when it was not possible to have both v4 and v6 VIPs at the same time.

Why is this important?
This is a followup to SDN-2213 ("Support dual ipv4 and ipv6 ingress and api VIPs").

We expect that customers with existing dual stack clusters will want to make use of the new dual stack VIPs fixes/enablement, but it's unclear how this will work because we've never supported modifying on-prem networking configuration after initial deployment. Once we have dual stack VIPs enabled, we will need to investigate how to alter the configuration to add VIPs to an existing cluster.

We will need to make changes to the VIP fields in the Infrastructure and/or ControllerConfig objects. Infrastructure would be the first option since that would make all of the fields consistent, but that relies on the ability to change that object and have the changes persist and be propagated to the ControllerConfig. If that's not possible, we may need to make changes just in ControllerConfig.

BU Priority Overview

Create custom roles for GCP with minimal set of required permissions.

Goals

Enable customers to better scope credential permissions and create custom roles on GCP that only include the minimum subset of what is needed for OpenShift.

State of the Business

Some of the service accounts that CCO creates, e.g. service account with role  roles/iam.serviceAccountUser provides elevated permissions that are not required/used by the requesting OpenShift components. This is because we use predefined roles for GCP that come with bunch of additional permissions. The goal is to create custom roles with only the required permissions. 

Execution Plans

TBD

Review the OVN Interconnect proposal, figure out the work that needs to be done in ovn-kubernetes to be able to move to this new OVN architecture. 

Phase-2 of this project in continuation of what was delivered in the earlier release. 

Why is this important?

OVN IC will be the model used in Hypershift. 

Goal

This goals of this features are:

• optimize and streamline the operations of HyperShift Operator (HO) on Azure Kubernetes Service (AKS) clusters
• Enable auto-detectopm of the underlying environment (managed or self-managed) to optimize the HO accordingly.

We drive OpenShift cross-market customer success and new customer adoption with constant improvements and feature additions to the existing capabilities of our OpenShift Core Networking (SDN, Network Edge, Network Observability). This feature captures that natural progression of the product for that development that does not align neatly to an existing Jira Feature.

Goals

• Feature enhancements (performance, scale, configuration, UX, ...)
• Modernization (incorporation and productization of new technologies)

Requirements

• Core Networking Stability
• Core Networking Performance and Scale
• Core Neworking Extensibility (e.g. Multus CNIs)
• Core Networking UX (Observability)
• Core Networking Security and Compliance

In Scope

• SDN (CNI plugins: openshift-sdn, ovn-kubernetes, network plumbing, network policy, bare metal networking, traffic routing, ...)

Out of Scope

There are grey areas, but in general:

• CNV
• Service Mesh
• CNF
• Telco-specific customer solutions (versus in-product features)

Documentation Considerations

Questions to be addressed:

• What educational or reference material (docs) is required to support this product feature? For users/admins? Other functions (security officers, etc)?
• Does this feature have doc impact?
• New Content, Updates to existing content, Release Note, or No Doc Impact
• If unsure and no Technical Writer is available, please contact Content Strategy.
• What concepts do customers need to understand to be successful in [action]?
• How do we expect customers will use the feature? For what purpose(s)?
• What reference material might a customer want/need to complete [action]?
• Is there source material that can be used as reference for the Technical Writer in writing the content? If yes, please link if available.
• What is the doc impact (New Content, Updates to existing content, or Release Note)?

Upstream K8s deprecated PodSecurityPolicy and replaced it with a new built-in admission controller that enforces the Pod Security Standards (See here for the motivations for deprecation).] There is an OpenShift-specific dedicated pod admission system called Security Context Constraints. Our aim is to keep the Security Context Constraints pod admission system while also allowing users to have access to the Kubernetes Pod Security Admission. 

With OpenShift 4.11, we are turned on the Pod Security Admission with global "privileged" enforcement. Additionally we set the "restricted" profile for warnings and audit. This configuration made it possible for users to opt-in their namespaces to Pod Security Admission with the per-namespace labels. We also introduced a new mechanism that automatically synchronizes the Pod Security Admission "warn" and "audit" labels.

With OpenShift 4.15, we intend to move the global configuration to enforce the "restricted" pod security profile globally. With this change, the label synchronization mechanism will also switch into a mode where it synchronizes the "enforce" Pod Security Admission label rather than the "audit" and "warn".