sphinx.addnodesdocument)}( rawsource children]docutils.nodessection)}(hhh](h title)}(h5OpenStack-based cloud infrastructure Testing Cookbookh]h Text5OpenStack-based cloud infrastructure Testing Cookbook}(hhparenth _documenthsourceNlineNuba
attributes}(ids]classes]names]dupnames]backrefs]utagnamehhhhhhP/var/jenkins_home/workspace/cntt-tox-rc1/doc/ref_cert/RC1/chapters/chapter04.rsthKubh)}(hhh](h)}(hIntroductionh]hIntroduction}(hh2hh0hhhNhNubah}(h!]h#]h%]h']h)]uh+hhh-hhhh,hKubh paragraph)}(h.Define the purpose of the chapter which is to:h]h.Define the purpose of the chapter which is to:}(hhBhh@hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKhh-hhubh bullet_list)}(hhh](h list_item)}(h1Identify Framework Needs, Goals, and Dependenciesh]h?)}(hhWh]h1Identify Framework Needs, Goals, and Dependencies}(hhWhhYhhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK hhUubah}(h!]h#]h%]h']h)]uh+hShhPhhhh,hNubhT)}(h:Define Opensource Integration (OVP, Functest, CVC, others)h]h?)}(hhnh]h:Define Opensource Integration (OVP, Functest, CVC, others)}(hhnhhphhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK
hhlubah}(h!]h#]h%]h']h)]uh+hShhPhhhh,hNubhT)}(h4Provide Automation Toolchain (list, topology, flow)
h]h?)}(h3Provide Automation Toolchain (list, topology, flow)h]h3Provide Automation Toolchain (list, topology, flow)}(hhhhhhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKhhubah}(h!]h#]h%]h']h)]uh+hShhPhhhh,hNubeh}(h!]h#]h%]h']h)]bullet-uh+hNhh,hK hh-hhubeh}(h!]introductionah#]h%]introductionah']h)]uh+h
hhhhhh,hKubh)}(hhh](h)}(h+Relevant Community Projects and Initiativesh]h+Relevant Community Projects and Initiatives}(hhhhhhhNhNubah}(h!]h#]h%]h']h)]uh+hhhhhhh,hKubh)}(hhh](h)}(hFunctesth]hFunctest}(hhhhhhhNhNubah}(h!]h#]h%]h']h)]uh+hhhhhhh,hKubh?)}(hXo `Functest `__ was
initially created to verify OPNFV Installers and Scenarios and then to
publish fair, trustable and public results regarding the status of the
different opensource technologies, especially for Neutron backends
(e.g. Neutron agents, OpenDaylight, OVN, etc.). It has been continuously
updated to offer the best testing coverage for any kind of OpenStack and
Kubernetes deployments including production environments. It also
ensures that the platforms meet Network Functions Virtualization
requirements by running and testing VNFs amongst all tests available.h](h reference)}(h>`Functest `__h]hFunctest}(hFunctesthhhhhNhNubah}(h!]h#]h%]h']h)]namehیrefuri/https://functest.readthedocs.io/en/stable-xena/uh+hhhubhX1 was
initially created to verify OPNFV Installers and Scenarios and then to
publish fair, trustable and public results regarding the status of the
different opensource technologies, especially for Neutron backends
(e.g. Neutron agents, OpenDaylight, OVN, etc.). It has been continuously
updated to offer the best testing coverage for any kind of OpenStack and
Kubernetes deployments including production environments. It also
ensures that the platforms meet Network Functions Virtualization
requirements by running and testing VNFs amongst all tests available.}(hX1 was
initially created to verify OPNFV Installers and Scenarios and then to
publish fair, trustable and public results regarding the status of the
different opensource technologies, especially for Neutron backends
(e.g. Neutron agents, OpenDaylight, OVN, etc.). It has been continuously
updated to offer the best testing coverage for any kind of OpenStack and
Kubernetes deployments including production environments. It also
ensures that the platforms meet Network Functions Virtualization
requirements by running and testing VNFs amongst all tests available.hhhhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hKhhhhubh?)}(hXZ Functest is driven by a true verification of the platform under test as
opposed to the interoperability programs such as
`RefStack `__ or `OPNFV Verification
Program `__ which select a small
subset of Functional tests passing in many different opensource software
combinations:h](hyFunctest is driven by a true verification of the platform under test as
opposed to the interoperability programs such as
}(hyFunctest is driven by a true verification of the platform under test as
opposed to the interoperability programs such as
hhhhhNhNubh)}(h.`RefStack `__h]hRefStack}(hRefStackhhhhhNhNubah}(h!]h#]h%]h']h)]namej hhttps://refstack.openstack.org/uh+hhhubh or }(h or hhhhhNhNubh)}(hC`OPNFV Verification
Program `__h]hOPNFV Verification
Program}(hOPNFV Verification
Programhj hhhNhNubah}(h!]h#]h%]h']h)]nameOPNFV Verification Programh"https://www.opnfv.org/verificationuh+hhhubhl which select a small
subset of Functional tests passing in many different opensource software
combinations:}(hl which select a small
subset of Functional tests passing in many different opensource software
combinations:hhhhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hKhhhhubhO)}(hhh](hT)}(hX; tests are skipped if an optional support is missing (e.g.
`Barbican `__ or
networking features such as `BGPVPN
interconnection `__
or `Service Function
Chaining `__)h]h?)}(hX; tests are skipped if an optional support is missing (e.g.
`Barbican `__ or
networking features such as `BGPVPN
interconnection `__
or `Service Function
Chaining `__)h](h:tests are skipped if an optional support is missing (e.g.
}(h:tests are skipped if an optional support is missing (e.g.
hj3 hhhNhNubh)}(h:`Barbican `__h]hBarbican}(hBarbicanhj< hhhNhNubah}(h!]h#]h%]h']h)]namejD h+https://docs.openstack.org/barbican/latest/uh+hhj3 ubh or
networking features such as }(h or
networking features such as hj3 hhhNhNubh)}(hQ`BGPVPN
interconnection `__h]hBGPVPN
interconnection}(hBGPVPN
interconnectionhjR hhhNhNubah}(h!]h#]h%]h']h)]nameBGPVPN interconnectionh4https://docs.openstack.org/networking-bgpvpn/latest/uh+hhj3 ubh
or }(h
or hj3 hhhNhNubh)}(hQ`Service Function
Chaining `__h]hService Function
Chaining}(hService Function
Chaininghji hhhNhNubah}(h!]h#]h%]h']h)]nameService Function Chainingh1https://docs.openstack.org/networking-sfc/latest/uh+hhj3 ubh)}(h)hj3 hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hK$hj/ ubah}(h!]h#]h%]h']h)]uh+hShj, hhhh,hNubhT)}(hCtests are parameterized (e.g. shared vs non-shared live migration)h]h?)}(hj h]hCtests are parameterized (e.g. shared vs non-shared live migration)}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK*hj ubah}(h!]h#]h%]h']h)]uh+hShj, hhhh,hNubhT)}(h-blacklist mechanisms are available if needed
h]h?)}(h,blacklist mechanisms are available if neededh]h,blacklist mechanisms are available if needed}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK+hj ubah}(h!]h#]h%]h']h)]uh+hShj, hhhh,hNubeh}(h!]h#]h%]h']h)]hhuh+hNhh,hK$hhhhubh?)}(hIt should be noted that `the RefStack
lists `__ are included as
they are in Functest in the next 3 dedicated testcases:h](hIt should be noted that }(hIt should be noted that hj hhhNhNubh)}(h8`the RefStack
lists `__h]hthe RefStack
lists}(hthe RefStack
listshj hhhNhNubah}(h!]h#]h%]h']h)]namethe RefStack listshhttps://refstack.openstack.org/uh+hhj ubhH are included as
they are in Functest in the next 3 dedicated testcases:}(hH are included as
they are in Functest in the next 3 dedicated testcases:hj hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hK-hhhhubhO)}(hhh](hT)}(h,refstack_compute (OpenStack Powered Compute)h]h?)}(hj h]h,refstack_compute (OpenStack Powered Compute)}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK1hj ubah}(h!]h#]h%]h']h)]uh+hShj hhhh,hNubhT)}(h2refstack_object (OpenStack Powered Object Storage)h]h?)}(hj h]h2refstack_object (OpenStack Powered Object Storage)}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK2hj ubah}(h!]h#]h%]h']h)]uh+hShj hhhh,hNubhT)}(h/refstack_platform (OpenStack Powered Platform)
h]h?)}(h.refstack_platform (OpenStack Powered Platform)h]h.refstack_platform (OpenStack Powered Platform)}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK3hj ubah}(h!]h#]h%]h']h)]uh+hShj hhhh,hNubeh}(h!]h#]h%]h']h)]hhuh+hNhh,hK1hhhhubh?)}(hXM Functest also integrates `Kubernetes End-to-end
tests `__
and allows verifying Kubernetes Conformance (see
`k8s-conformance `__).h](hFunctest also integrates }(hFunctest also integrates hj6 hhhNhNubh)}(hs`Kubernetes End-to-end
tests `__h]hKubernetes End-to-end
tests}(hKubernetes End-to-end
testshj? hhhNhNubah}(h!]h#]h%]h']h)]nameKubernetes End-to-end testshQhttps://kubernetes.io/blog/2019/03/22/kubernetes-end-to-end-testing-for-everyone/uh+hhj6 ubh2
and allows verifying Kubernetes Conformance (see
}(h2
and allows verifying Kubernetes Conformance (see
hj6 hhhNhNubh)}(h`k8s-conformance `__h]hk8s-conformance}(hk8s-conformancehjV hhhNhNubah}(h!]h#]h%]h']h)]namej^ hwhttps://build.opnfv.org/ci/job/functest-kubernetes-opnfv-functest-kubernetes-smoke-v1.22-k8s_conformance-run/25/consoleuh+hhj6 ubh).}(h).hj6 hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hK5hhhhubh?)}(hX Dovetail (OVP) mostly leverages on Functest but only runs a small part
of Functest (~15% of all functional tests, no benchmarking tests, no VNF
deployment and testing). It’s worth mentioning that Functest is patched
to `disable API
verification `__
which has differed from OpenStack rules for years.h](hDovetail (OVP) mostly leverages on Functest but only runs a small part
of Functest (~15% of all functional tests, no benchmarking tests, no VNF
deployment and testing). It’s worth mentioning that Functest is patched
to }(hDovetail (OVP) mostly leverages on Functest but only runs a small part
of Functest (~15% of all functional tests, no benchmarking tests, no VNF
deployment and testing). It’s worth mentioning that Functest is patched
to hjr hhhNhNubh)}(hx`disable API
verification `__h]hdisable API
verification}(hdisable API
verificationhj{ hhhNhNubah}(h!]h#]h%]h']h)]namedisable API verificationhYhttps://github.com/opnfv/dovetail/tree/master/etc/patches/functest/disable-api-validationuh+hhjr ubh3
which has differed from OpenStack rules for years.}(h3
which has differed from OpenStack rules for years.hjr hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hK:hhhhubh?)}(hX Then Functest conforms with the upstream rules (versions, code quality,
etc.) and especially their
`gates `__
(a.k.a. the automatic verification prior to any code review) to preserve
the quality between code and deployment. In that case, Functest can be
considered as a smooth and lightweight integration of tests developed
upstream (and the Functest team directly contributes in these projects:
`Rally `__,
`Tempest `__, etc.). It’s worth
mentioning that, as opposed to the OpenStack Gates leveraging on
`DevStack `__, it can check
the same already deployed SUT over and over even from a `Raspberry
PI `__. Here the testcases can be executed
in parallel vs the same deployment instead of being executed vs
different pools of virtual machines.h](hcThen Functest conforms with the upstream rules (versions, code quality,
etc.) and especially their
}(hcThen Functest conforms with the upstream rules (versions, code quality,
etc.) and especially their
hj hhhNhNubh)}(hM`gates `__h]hgates}(hgateshj hhhNhNubah}(h!]h#]h%]h']h)]namej hAhttps://docs.openstack.org/infra/system-config/devstack-gate.htmluh+hhj ubhX
(a.k.a. the automatic verification prior to any code review) to preserve
the quality between code and deployment. In that case, Functest can be
considered as a smooth and lightweight integration of tests developed
upstream (and the Functest team directly contributes in these projects:
}(hX
(a.k.a. the automatic verification prior to any code review) to preserve
the quality between code and deployment. In that case, Functest can be
considered as a smooth and lightweight integration of tests developed
upstream (and the Functest team directly contributes in these projects:
hj hhhNhNubh)}(h8`Rally `__h]hRally}(hRallyhj hhhNhNubah}(h!]h#]h%]h']h)]namej h,https://github.com/openstack/rally-openstackuh+hhj ubh,
}(h,
hj hhhNhNubh)}(h2`Tempest `__h]hTempest}(hTempesthj hhhNhNubah}(h!]h#]h%]h']h)]namej h$https://github.com/openstack/tempestuh+hhj ubhW, etc.). It’s worth
mentioning that, as opposed to the OpenStack Gates leveraging on
}(hW, etc.). It’s worth
mentioning that, as opposed to the OpenStack Gates leveraging on
hj hhhNhNubh)}(h:`DevStack `__h]hDevStack}(hDevStackhj hhhNhNubah}(h!]h#]h%]h']h)]namej h+https://docs.openstack.org/devstack/latest/uh+hhj ubhG, it can check
the same already deployed SUT over and over even from a }(hG, it can check
the same already deployed SUT over and over even from a hj hhhNhNubh)}(h/`Raspberry
PI `__h]hRaspberry
PI}(hRaspberry
PIhj hhhNhNubah}(h!]h#]h%]h']h)]nameRaspberry PIhhttps://www.raspberrypi.org/uh+hhj ubh. Here the testcases can be executed
in parallel vs the same deployment instead of being executed vs
different pools of virtual machines.}(h. Here the testcases can be executed
in parallel vs the same deployment instead of being executed vs
different pools of virtual machines.hj hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hKAhhhhubh?)}(hHere are the functional tests (>2000) running in OpenStack gates
integrated in Functest Smoke (see `Functest daily
jobs `__
for more details):h](hcHere are the functional tests (>2000) running in OpenStack gates
integrated in Functest Smoke (see }(hcHere are the functional tests (>2000) running in OpenStack gates
integrated in Functest Smoke (see hj hhhNhNubh)}(ha`Functest daily
jobs `__h]hFunctest daily
jobs}(hFunctest daily
jobshj hhhNhNubah}(h!]h#]h%]h']h)]nameFunctest daily jobshGhttps://build.opnfv.org/ci/view/functest/job/functest-wallaby-daily/17/uh+hhj ubh
for more details):}(h
for more details):hj hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hKQhhhhubh table)}(hhh]h tgroup)}(hhh](h colspec)}(hhh]h}(h!]h#]h%]h']h)]colwidthKuh+jF hjC ubjG )}(hhh]h}(h!]h#]h%]h']h)]colwidthKuh+jF hjC ubh thead)}(hhh]h row)}(hhh](h entry)}(hhh]h?)}(h Testcasesh]h Testcases}(hjm hjk hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKWhjh ubah}(h!]h#]h%]h']h)]uh+jf hjc ubjg )}(hhh]h?)}(hGatesh]hGates}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKWhj ubah}(h!]h#]h%]h']h)]uh+jf hjc ubeh}(h!]h#]h%]h']h)]uh+ja hj^ ubah}(h!]h#]h%]h']h)]uh+j\ hjC ubh tbody)}(hhh](jb )}(hhh](jg )}(hhh]h?)}(htempest_neutronh]htempest_neutron}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKYhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hNeutronh]hNeutron}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKYhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(htempest_cinderh]htempest_cinder}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKZhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hCinderh]hCinder}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKZhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(htempest_keystoneh]htempest_keystone}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK[hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hKeystoneh]hKeystone}(hj4 hj2 hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK[hj/ ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(hrally_sanityh]hrally_sanity}(hjT hjR hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK\hjO ubah}(h!]h#]h%]h']h)]uh+jf hjL ubjg )}(hhh]h?)}(hGeneralh]hGeneral}(hjk hji hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK\hjf ubah}(h!]h#]h%]h']h)]uh+jf hjL ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(hrefstack_defcoreh]hrefstack_defcore}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK]hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hGeneralh]hGeneral}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK]hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(htempest_fullh]htempest_full}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK^hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hGeneralh]hGeneral}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK^hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(htempest_slowh]htempest_slow}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK_hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hGeneralh]hGeneral}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK_hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(htempest_scenarioh]htempest_scenario}(hj0 hj. hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK`hj+ ubah}(h!]h#]h%]h']h)]uh+jf hj( ubjg )}(hhh]h?)}(hGeneralh]hGeneral}(hjG hjE hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK`hjB ubah}(h!]h#]h%]h']h)]uh+jf hj( ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(hpatroleh]hpatrole}(hjg hje hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKahjb ubah}(h!]h#]h%]h']h)]uh+jf hj_ ubjg )}(hhh]h?)}(hPatroleh]hPatrole}(hj~ hj| hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKahjy ubah}(h!]h#]h%]h']h)]uh+jf hj_ ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(htempest_barbicanh]htempest_barbican}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKbhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hBarbicanh]hBarbican}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKbhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(hnetworking-bgpvpnh]hnetworking-bgpvpn}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKchj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hNetworking BGP VPNh]hNetworking BGP VPN}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKchj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(hnetworking-sfch]hnetworking-sfc}(hj hj
hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKdhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hNetworking SFCh]hNetworking SFC}(hj# hj! hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKdhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubeh}(h!]h#]h%]h']h)]uh+j hjC ubeh}(h!]h#]h%]h']h)]colsKuh+jA hj> ubah}(h!]h#]h%]h']h)]uh+j< hhhhhh,hNubh?)}(hTo complete functional testing, Functest also integrates a few
`performance
tools `__
(2-3 hours) as proposed by OpenStack:h](h?To complete functional testing, Functest also integrates a few
}(h?To complete functional testing, Functest also integrates a few
hjN hhhNhNubh)}(hf`performance
tools `__h]hperformance
tools}(hperformance
toolshjW hhhNhNubah}(h!]h#]h%]h']h)]nameperformance toolshNhttps://docs.openstack.org/developer/performance-docs/methodologies/tools.htmluh+hhjN ubh&
(2-3 hours) as proposed by OpenStack:}(h&
(2-3 hours) as proposed by OpenStack:hjN hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hKghhhhubj= )}(hhh]jB )}(hhh](jG )}(hhh]h}(h!]h#]h%]h']h)]colwidthK
uh+jF hjw ubjG )}(hhh]h}(h!]h#]h%]h']h)]colwidthKuh+jF hjw ubj] )}(hhh]jb )}(hhh](jg )}(hhh]h?)}(h Testcasesh]h Testcases}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKmhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hBenchmarkingh]hBenchmarking}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKmhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubah}(h!]h#]h%]h']h)]uh+j\ hjw ubj )}(hhh](jb )}(hhh](jg )}(hhh]h?)}(h
rally_fullh]h
rally_full}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKohj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hControl Plane (API) testingh]hControl Plane (API) testing}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKohj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(h
rally_jobsh]h
rally_jobs}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKphj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hControl Plane (API) testingh]hControl Plane (API) testing}(hj' hj% hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKphj" ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(hvmtph]hvmtp}(hjG hjE hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKqhjB ubah}(h!]h#]h%]h']h)]uh+jf hj? ubjg )}(hhh]h?)}(hData Plane testingh]hData Plane testing}(hj^ hj\ hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKqhjY ubah}(h!]h#]h%]h']h)]uh+jf hj? ubeh}(h!]h#]h%]h']h)]uh+ja hj ubjb )}(hhh](jg )}(hhh]h?)}(hshakerh]hshaker}(hj~ hj| hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKrhjy ubah}(h!]h#]h%]h']h)]uh+jf hjv ubjg )}(hhh]h?)}(hData Plane testingh]hData Plane testing}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKrhj ubah}(h!]h#]h%]h']h)]uh+jf hjv ubeh}(h!]h#]h%]h']h)]uh+ja hj ubeh}(h!]h#]h%]h']h)]uh+j hjw ubeh}(h!]h#]h%]h']h)]colsKuh+jA hjt ubah}(h!]h#]h%]h']h)]uh+j< hhhhhh,hNubh?)}(h,And VNFs automatically deployed and tested :h]h,And VNFs automatically deployed and tested :}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKuhhhhubj= )}(hhh]jB )}(hhh](jG )}(hhh]h}(h!]h#]h%]h']h)]colwidthKuh+jF hj ubjG )}(hhh]h}(h!]h#]h%]h']h)]colwidthK#uh+jF hj ubj] )}(hhh]jb )}(hhh](jg )}(hhh]h?)}(h Testcasesh]h Testcases}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKxhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hBenchmarkingh]hBenchmarking}(hj
hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKxhj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj ubah}(h!]h#]h%]h']h)]uh+j\ hj ubj )}(hhh](jb )}(hhh](jg )}(hhh]h?)}(hcloudifyh]hcloudify}(hj3 hj1 hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKzhj. ubah}(h!]h#]h%]h']h)]uh+jf hj+ ubjg )}(hhh]h?)}(hCloudify deploymenth]hCloudify deployment}(hjJ hjH hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKzhjE ubah}(h!]h#]h%]h']h)]uh+jf hj+ ubeh}(h!]h#]h%]h']h)]uh+ja hj( ubjb )}(hhh](jg )}(hhh]h?)}(hcloudify_imsh]hcloudify_ims}(hjj hjh hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK{hje ubah}(h!]h#]h%]h']h)]uh+jf hjb ubjg )}(hhh]h?)}(h#Clearwater IMS deployed via Coudifyh]h#Clearwater IMS deployed via Coudify}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK{hj| ubah}(h!]h#]h%]h']h)]uh+jf hjb ubeh}(h!]h#]h%]h']h)]uh+ja hj( ubjb )}(hhh](jg )}(hhh]h?)}(hheat_imsh]hheat_ims}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK|hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(h Clearwater IMS deployed via Heath]h Clearwater IMS deployed via Heat}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK|hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj( ubjb )}(hhh](jg )}(hhh]h?)}(hvyos_vrouterh]hvyos_vrouter}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK}hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hVyOS deployed via Cloudifyh]hVyOS deployed via Cloudify}(hj hj hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK}hj ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj( ubjb )}(hhh](jg )}(hhh]h?)}(hjuju_epch]hjuju_epc}(hj hj
hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK~hj
ubah}(h!]h#]h%]h']h)]uh+jf hj ubjg )}(hhh]h?)}(hOAI deployed via Jujuh]hOAI deployed via Juju}(hj& hj$ hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hK~hj! ubah}(h!]h#]h%]h']h)]uh+jf hj ubeh}(h!]h#]h%]h']h)]uh+ja hj( ubeh}(h!]h#]h%]h']h)]uh+j hj ubeh}(h!]h#]h%]h']h)]colsKuh+jA hj ubah}(h!]h#]h%]h']h)]uh+j< hhhhhh,hNubh?)}(hlFunctest should be considered as a whole as it meets multiple objectives
about the reference implementation:h]hlFunctest should be considered as a whole as it meets multiple objectives
about the reference implementation:}(hjS hjQ hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKhhhhubhO)}(hhh](hT)}(h\verify all APIs (services, advances, features, etc.) exposed by the
reference implementationh]h?)}(h\verify all APIs (services, advances, features, etc.) exposed by the
reference implementationh]h\verify all APIs (services, advances, features, etc.) exposed by the
reference implementation}(hjh hjf hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKhjb ubah}(h!]h#]h%]h']h)]uh+hShj_ hhhh,hNubhT)}(hcompare the reference implementation and local deployments from a
functional standpoint and from OpenStack control plane and dataplane
capabilities
h]h?)}(hcompare the reference implementation and local deployments from a
functional standpoint and from OpenStack control plane and dataplane
capabilitiesh]hcompare the reference implementation and local deployments from a
functional standpoint and from OpenStack control plane and dataplane
capabilities}(hj hj~ hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKhjz ubah}(h!]h#]h%]h']h)]uh+hShj_ hhhh,hNubeh}(h!]h#]h%]h']h)]hhuh+hNhh,hKhhhhubh?)}(hX It’s worth mentioning that Functest already takes into account the first
Anuket
`profiles `__.
Anuket should simply add the next Functest inputs according the
reference implementation:h](hRIt’s worth mentioning that Functest already takes into account the first
Anuket
}(hRIt’s worth mentioning that Functest already takes into account the first
Anuket
hj hhhNhNubh)}(hS`profiles `__h]hprofiles}(hprofileshj hhhNhNubah}(h!]h#]h%]h']h)]namej hDhttps://git.opnfv.org/functest/tree/functest/ci/config_patch.yaml#n2uh+hhj ubh[.
Anuket should simply add the next Functest inputs according the
reference implementation:}(h[.
Anuket should simply add the next Functest inputs according the
reference implementation:hj hhhNhNubeh}(h!]h#]h%]h']h)]uh+h>hh,hKhhhhubhO)}(hhh](hT)}(hc`Functest
inputs `__h]h?)}(hc`Functest
inputs `__h]h)}(hj h]hFunctest
inputs}(hFunctest
inputshj hhhNhNubah}(h!]h#]h%]h']h)]nameFunctest inputshMhttps://github.com/opnfv/functest/blob/stable/iruya/functest/utils/env.py#L17uh+hhj ubah}(h!]h#]h%]h']h)]uh+h>hh,hKhj ubah}(h!]h#]h%]h']h)]uh+hShj hhhh,hNubhT)}(h`tempest specific
configuration `__
h]h?)}(h`tempest specific
configuration `__h]h)}(hj h]htempest specific
configuration}(htempest specific
configurationhj hhhNhNubah}(h!]h#]h%]h']h)]nametempest specific configurationhyhttps://github.com/opnfv/functest/blob/stable/iruya/functest/opnfv_tests/openstack/tempest/custom_tests/tempest_conf.yamluh+hhj ubah}(h!]h#]h%]h']h)]uh+h>hh,hKhj ubah}(h!]h#]h%]h']h)]uh+hShj hhhh,hNubeh}(h!]h#]h%]h']h)]hhuh+hNhh,hKhhhhubh?)}(hAdditional links:h]hAdditional links:}(hj
hj
hhhNhNubah}(h!]h#]h%]h']h)]uh+h>hh,hKhhhhubhO)}(hhh](hT)}(h?`Homepage `__h]h?)}(hj#
h]h)}(hj#
h]hHomepage}(hHomepagehj(
hhhNhNubah}(h!]h#]h%]h']h)]namej/
h0https://functest.readthedocs.io/en/stable-iruya/uh+hhj%
ubah}(h!]h#]h%]h']h)]uh+h>hh,hKhj!
ubah}(h!]h#]h%]h']h)]uh+hShj
hhhh,hNubhT)}(hi`Run Alpine Functest containers
(Iruya) `__h]h?)}(hi`Run Alpine Functest containers
(Iruya) `__h]h)}(hjJ
h]h&Run Alpine Functest containers
(Iruya)}(h&Run Alpine Functest containers
(Iruya)hjL
hhhNhNubah}(h!]h#]h%]h']h)]name&Run Alpine Functest containers (Iruya)h