6. EPMO Open Source Coordination Office Redaction File Detail Report

Produced by Araxis Merge on 12/9/2016 1:23:19 PM Central Standard Time. See www.araxis.com for information about Merge. This report uses XHTML and CSS2, and is best viewed with a modern standards-compliant browser. For optimum results when printing this report, use landscape orientation and enable printing of background images and colours in your browser.

6.1 Files compared

#	Location	File	Last Modified
1	C:\Users\vhaisbforrez\AraxisComp\PUB_UN\EPIP_Test_Cases_Functional Testing_(OR_3.0_431)_201611.zip	EPIP_Test_Evaluation_(OR_3.0_431)_201611.docx	Wed Dec 7 19:58:00 2016 UTC
2	C:\Users\vhaisbforrez\AraxisComp\PUB_RE\EPIP_Test_Cases_Functional Testing_(OR_3.0_431)_201611.zip	EPIP_Test_Evaluation_(OR_3.0_431)_201611.docx	Thu Dec 8 21:39:58 2016 UTC

6.2 Comparison summary

Description	Between Files 1 and 2
Description	Text Blocks	Lines
Unchanged	1	514
Changed	0	0
Inserted	0	0
Removed	0	0

6.3 Comparison options

Whitespace
Character case	Differences in character case are significant
Line endings	Differences in line endings (`CR` and `LF` characters) are ignored
CR/LF characters	Not shown in the comparison detail

6.4 Active regular expressions

No regular expressions were active.

6.5 Comparison detail

	1	Existing P roduct Int ake Progra m (EPIP)
	2	Patch OR3 .0431
	3	Test Evalu ation
	4
	5
	6
	7	Department of Vetera ns Affairs
	8	November 2 016
	9	Version 1. 0
	10
	11
	12
	13	Revision H istory
	14	Note: The revision h istory cyc le begins once chang es or enha ncements a re request ed after t he Communi cations Pl an has bee n baseline d.
	15	Date
	16	Version
	17	Descriptio n
	18	Author
	19	11/18/2016
	20	1.0
	21	Initial do cument.
	22	EPIP Proje ct Team
	23
	24	Artifact R ationale
	25	The test e valuation document i s the prim ary output of the te st and eva luation pr ocess, an integral p art of the systems e ngineering process, which iden tifies lev els of per formance a nd assists the devel oper in co rrecting d eficiencie s.
	26
	27	Table of C ontents
	28	1.Test Eva luation In troduction 1
	29	1.1.Test E valuation Scope1
	30	1.2.Test A rchitectur e1
	31	1.3.Test E nvironment /Configura tion1
	32	1.4.Instal lation Pro cess2
	33	2.Test Dat a2
	34	3.Issues2
	35	4.Test Exe cution Log 2
	36	5.Test Def ect Log3
	37	6.Test Res ults Summa ry3
	38	6.1.Defect Severity and Priori ty Levels3
	39	6.2.Total Defects by Severity Level3
	40	6.3.Breakd own of Tes t Results4
	41	6.4.Perfor mance Test ing4
	42	7.Test Cov erage4
	43	7.1.Requir ements Cov ered4
	44	7.2.Sectio n 508 Comp liance Cov erage4
	45	8.Suggeste d Actions4
	46	9.Defect S everity an d Priority Definitio ns5
	47	9.1.Defect Severity Level5
	48	9.1.1.Seve rity Level 1 – Criti cal5
	49	9.1.2.Seve rity Level 2 - High5
	50	9.1.3.Seve rity Level 3 - Mediu m6
	51	9.1.4.Seve rity Level 4 - Low6
	52	9.2.Priori ty Classif ications6
	53	9.2.1.Prio rity 1 - R esolve Imm ediately6
	54	9.2.2.Prio rity 2 - G ive High A ttention6
	55	9.2.3.Prio rity 3 - N ormal Queu e7
	56	9.2.4.Prio rity 4 - L ow Priorit y7
	57	10.Optiona l Tables, Charts, an d Graphs7
	58	11.Documen t Approval Signature s8
	59	Appendix A - Test Ex ecution Lo g9
	60	Appendix B – Defect Log10
	61
	62	Test Evalu ation Intr oduction
	63	The purpos e of this Test Evalu ation is t o:
	64	Identify t he testing approach used.
	65	Present a summary an alysis of the key te st results from the remediatio n of this intake for review an d assessme nt by desi gnated sta keholders.
	66	Provide a general st atement of the quali ty of the system und er test.
	67	Make recom mendations for futur e testing efforts.
	68	Test Evalu ation Scop e
	69	The scope of this Te st Evaluat ion is to verify the functiona lity of th e Patch OR 3.0431 c ode modifi cation, as determine d by Funct ional, Com ponent Int egration/S ystem, and Regressio n testing. Testing a ctivities followed t he specifi cations ou tlined in the follow ing Master Test Plan : OR3.04 31 Master Test Plan (included in Appendi x A).
	70	Test Archi tecture
	71	Following are the EP IP test ac counts use d by the L eidos Deve lopment an d SQA team s to test OR3.0431 .
	72	Developmen t Test Acc ounts
	73	(for Unit Testing)
	74	SQA Test A ccounts
	75	(for Funct ional, Reg ression, a nd Compone nt Integra tion and S ystem Test ing)
	76	VistAS1 ( alternate name: D1S1 )
	77	VistAG1 (a lternate n ame: D1G1)
	78	VistAS2 ( alternate name: D1S2 )
	79	VistAG2 (a lternate n ame: D1G2) –for CPRS GUI testi ng only
	80	Test Envir onment/Con figuration
	81	The EPIP t est accoun ts are mai ntained by the EPIP System Adm inistrator , who inst alls all V A-released patches a s soon as they are n ationally released. All EPIP t est accoun ts are clo ned from e xisting VA Enterpris e Testing Services ( ETS) test accounts. The Comput erized Pat ient Recor d System ( CPRS) Grap hical User Interface (GUI) exe cutable is configure d for each VistA ins tance util izing a un ique Inter net Protoc ol (IP) ad dress to c onnect to the VistA applicatio ns. Any up dates to t he CPRS GU I executab le are han dled by th e EPIP Sys tem Admini strator.
	82	All EPIP T est Engine ers and De velopers w ho have th e proper c redentials can acces s the test accounts. The VA Au stin Infor mation Tec hnology Ce nter (AITC ) support team reset s password s and sets up new ac cess crede ntials on an as-need ed basis.
	83	Installati on Process
	84	As soon as the remed iation pro cess is co mplete and the patch is availa ble for te sting, a K IDS build is created in the De velopment account an d then sen t to FORUM for final packaging . The patc h is then submitted to the VA SQA Lead’s Mailman a ccount for installat ion.
	85	An EPIP De veloper or Test Engi neer utili zes the KI DS Install ation proc ess to ext ract the b uild from the patch and instal l the buil d into a t est accoun t. The ind ividual wh o installs the patch verifies the routin e checksum s and also checks fo r errors d uring the installati on process . If the p atch is su ccessfully installed without a ny errors, then the EPIP Test team proce eds with F unctional, Regressio n, and Com ponent Int egration a nd System testing. I f defects are found, then the Developmen t team wor ks to find a resolut ion and cr eates new versions o f the patc h until al l defects are resolv ed.
	86	Test Data
	87	The EPIP T est team u tilizes th e test dat a in the d esignated test accou nts (D1G1, D1G2).
	88	The test d ata is enc rypted fol lowing the standards set forth by the VA Office of Informati on & Techn ology (OI& T). All Pe rsonally I dentifiabl e Informat ion (PII) and Protec ted Health Informati on (PHI) i s scrubbed and is no t availabl e to the T est Engine ers.
	89	All testin g is execu ted using encrypted test patie nts availa ble from a ny of the EPIP test accounts. Examples o f encrypte d test pat ients:
	90	AAAHURMMX, XPHY
	91	BADHB, HAA DXS
	92	FDHUX, YHI J
	93	All tests were execu ted manual ly by EPIP Test Engi neers.
	94	Issues
	95	No issues were encou ntered dur ing testin g of OR3. 0431.
	96	Title
	97	Issue Desc ription
	98	Type
	99	Severity
	100	N/A
	101	N/A
	102	N/A
	103	N/A
	104	Test Execu tion Log
	105	The Test E xecution L og records the execu tion of te st scripts and docum ents the t est result s for each test scri pt. The Te st Enginee r is respo nsible for completin g the Test Execution Log.
	106	The Test E xecution R ecords for OR3.043 1 are incl uded in th e EPIP Pat ch OR3.0 431 Master Test Plan . The Mast er Test Pl an is avai lable in A ppendix A.
	107
	108	Test Defec t Log
	109	The Test D efect Log is a tool for record ing, analy zing, trac king, and documentin g the clos ure of def ects. It s pecifies t he Defect ID, affect ed applica tion, beha vior or re sult that occurred, and the IE EE-defined Severity Level. It includes e nough info rmation fo r the deve loper to f ind and re -create th e defect. The Defect Log is av ailable in Appendix B.
	110	Test Resul ts Summary
	111	NSR2008031 7 Default Encounter Location – SQA testi ng for thi s NSR star ted in the Dev1 Gold 1 test env ironment o n Septembe r 19, 2016 and ended on Novemb er 16, 201 6. Test ve rsion 1 wa s installe d in the t est enviro nment afte r Unit tes ting was c ompleted. Upon compl etion of I ntegration testing ( Component Integratio n and Syst em Testing , Function al Testing , and Regr ession Tes ting), one (1) defec t was foun d and repo rted. This defect wa s resolved with vers ion 2 of t he patch. The new ve rsion was retested b y the Test Engineer and no oth er defects were foun d for this functiona lity.
	112	NSR2015060 8 Accessio n of Site- Supported Lab Test – SQA test ing for th is NSR sta rted in th e Dev1 Gol d1 test en vironment on Septemb er 19, 201 6 and ende d on Novem ber 16, 20 16. Test v ersion 1 w as install ed in the test envir onment aft er Unit te sting was completed. Upon comp letion of Integratio n testing (Component Integrati on and Sys tem Testin g, Functio nal Testin g, and Reg ression Te sting), ze ro (0) def ects were found and reported.
	113	NSR2014121 0 CPRS Pop -Up Box – SQA testin g for this NSR start ed in the Dev1 Gold1 test envi ronment on September 19, 2016 and ended on Novembe r 16, 2016 . Test ver sion 1 was installed in the te st environ ment after Unit test ing was co mpleted. U pon comple tion of In tegration testing (C omponent I ntegration and Syste m Testing, Functiona l Testing, and Regre ssion Test ing), five (5) defec ts were fo und and re ported. Al l five def ects were resolved w ith new ve rsions of the patch and retest ed by the Test Engin eer. No ot her defect s were fou nd for thi s function ality.
	114	Defect Sev erity and Priority L evels
	115	A defect i s defined as a flaw in a compo nent or sy stem that can cause the compon ent or sys tem to fai l to perfo rm its req uired func tion, e.g. , an incor rect state ment or da ta definit ion. A def ect, if en countered during exe cution, ma y cause a failure of the compo nent or sy stem.
	116	Defects ar e categori zed accord ing to sev erity and priority l evels. The test anal yst assign s the seve rity, whil e the deve lopment ma nager assi gns the pr iority for repair. F or more in formation, see Defec t Severity and Prior ity Defini tion in th is Test Ev aluation.
	117	Total Defe cts by Sev erity Leve l
	118	The follow ing defect s were fou nd during testing of Test vers ion 1.
	119	Defect Num ber
	120	Severity
	121	419696
	122	Medium
	123	419712
	124	Medium
	125	419795
	126	Medium
	127	419989
	128	High
	129	419992
	130	Medium
	131	419996
	132	Medium
	133	Breakdown of Test Re sults
	134	Testing wa s complete d on Nove mber 16, 2 016. All t est result s were rec orded in t he Rationa l Quality Manager to ol. Detail ed results are avail able in th e EPIP Pat ch OR3.0 431 Master Test Plan (see Appe ndix A).
	135	Performanc e Testing
	136	Performanc e testing was not co nducted.
	137	Test Cover age
	138	The EPIP P atch OR3. 0431 Mast er Test Pl an contain s details on test co verage (se e Appendix A).
	139	Requiremen ts Covered
	140	The follow ing Tracea bility Mat rix for Pa tch OR3.0 431 repor ts informa tion store d in the R ational to ol, such a s the tota l number o f requirem ents and t he percent age of req uirements covered by executed tests.
	141
	142	Section 50 8 Complian ce Coverag e
	143	Section 50 8 testing was not re quired for this patc h.
	144	Suggested Actions
	145	None.
	146	Defect Sev erity and Priority D efinitions
	147	The classi fication o f defects within a s ystem exam ines both the severi ty and pri ority of t he defect.
	148	Severity i s a measur e of how g reat the i mpact is o n the user ’s ability to comple te the doc umented ac tions with in the sys tem.
	149	Priority d etermines the speed with which a given d efect must be repair ed.
	150	Defect cla ssificatio n may be d etermined either bec ause testi ng is dela yed by a f ailure in the system or becaus e a cumber some worka round prev ents a use r from com pleting th e assigned tasks. Bo th severit y and prio rity measu res must b e recorded when sche duling def ect resolu tion tasks .
	151	Defect Sev erity Leve l
	152	The follow ing subsec tions iden tify the d efect seve rity level s.
	153	Severity L evel 1 – C ritical
	154	Institute of Electri cal and El ectronics Engineers (IEEE) def inition: T he defect results in the failu re of the complete s oftware sy stem, of a subsystem , or of a software u nit (progr am or modu le) within the syste m.
	155	Any defect that comp romises pa tient safe ty or syst em securit y. Example s of syste m security defects i nclude bre ach of con fidentiali ty require ments of t he Privacy Act, the Health Ins urance Por tability a nd Account ability Ac t (HIPAA), or Federa l Tax Info rmation gu idelines.
	156	Loss of sy stem funct ionality c ritical to user oper ations wit h no suita ble workar ound, i.e. , there is no way to achieve t he expecte d results using the applicatio n.
	157	System cra sh or hang that prev ents furth er testing or operat ion of the complete applicatio n or a sec tion of th e applicat ion.
	158	Any defect that caus es corrupt ion of dat a from a r esult of t he system (as oppose d to user error).
	159	Any defect in which inappropri ate transm issions ar e consiste ntly gener ated or ap propriate transmissi ons of HL7 messages fail to be generated .
	160	Loss of fu nctionalit y resultin g in erron eous eligi bility/enr ollment de terminatio ns or comm unications not being sent.
	161	Severity L evel 2 - H igh
	162	IEEE defin ition: The defect re sults in t he failure of the co mplete sof tware syst em, of a s ubsystem, or of a so ftware uni t (program or module ) within t he system. There is no way to make the f ailed comp onent(s) f unction. H owever, th ere are ac ceptable p rocessing alternativ es which w ill yield the desire d result.
	163	A major de fect in th e function ality that does not result in corruption of data.
	164	A major de fect in th e function ality resu lting in a failure o f all or p art of the applicati on, where:
	165	The expect ed results can tempo rarily be achieved b y alternat e means. T he custome r indicate s the work around is acceptabl e for the short term .
	166	Any defect that does not confo rm to Sect ion 508 st andards.
	167	Any defect that resu lts in ina ccurate or missing r equirement s.
	168	Any defect that resu lts in inv alid authe ntication or authent ication of an invali d end user .
	169	Severity L evel 3 - M edium
	170	IEEE defin ition: The defect do es not res ult in a f ailure, bu t causes t he system to produce incorrect , incomple te, or inc onsistent results, o r the defe ct impairs the syste ms usabili ty.
	171	Minor func tionality is not wor king as in tended and a workaro und exists but is no t suitable for long term use
	172	The inabil ity of a v alid user to access the system consisten t with gra nted privi leges
	173	Typographi cal or gra mmatical e rrors in t he applica tion, incl uding inst allation g uides, use r guides, training m anuals, an d design d ocuments
	174	Any defect producing cryptic, incorrect, or inappr opriate er ror messag es
	175	Any defect that resu lts from t he use of non-standa rd data te rminology in the app lication o r document ation, as defined by the Depar tment of V eterans Af fairs
	176	Cosmetic i ssues that are impor tant to th e integrit y of the p roduct, bu t do not r esult in d ata entry and or dat a quality problems.
	177	Severity L evel 4 - L ow
	178	IEEE defin ition: The defect do es not cau se a failu re, does n ot impair usability, and the d esired pro cessing re sults are easily obt ained by w orking aro und the de fect.
	179	Minor loss of, or de fect in th e function ality wher e a long t erm use ex ists
	180	Low-level cosmetic i ssues.
	181	Priority C lassificat ions
	182	The follow ing subsec tions iden tify the a ppropriate actions f or defects at each p riority le vel, per d efinitions of IEEE.
	183	Priority 1 - Resolve Immediate ly
	184	Further de velopment and/or tes ting canno t occur un til the de fect has b een repair ed. The sy stem canno t be used until the repair has been affe cted.
	185	Priority 2 - Give Hi gh Attenti on
	186	The defect must be r esolved as soon as p ossible be cause it i s impairin g developm ent and/or testing a ctivities. System us e will be severely a ffected un til the de fect is fi xed.
	187	Priority 3 - Normal Queue
	188	The defect should be resolved in the nor mal course of develo pment acti vities. It can wait until a ne w build or version i s created.
	189	Priority 4 - Low Pri ority
	190	The defect is an irr itant that should be repaired, but can b e repaired after mor e serious defects ha ve been fi xed.
	191	Optional T ables, Cha rts, and G raphs
	192	None.
	193	Document A pproval Si gnatures
	194
	195	Signed: __ __________ __________ __________ __________ __________ __________ _________
	196	Program/P roject Man agerDate
	197
	198	Signed: __ __________ __________ __________ __________ __________ __________ _________
	199	Business Sponsor Re presentati veDate
	200
	201	Signed: __ __________ __________ __________ __________ __________ __________ _________
	202	Test Lead Date
	203
	204	Appendix A - Test Ex ecution Lo g
	205	Please ref er to the attached O R3.0431 Master Tes t Plan to view the T est Execut ion Record s.
	206
	207	Appendix B – Defect Log
	208	Six defect s were fou nd during testing of OR3.043 1.
	209	Defect ID
	210	Patch Numb er
	211	Affected A pplication
	212	NSR Number
	213	Severity
	214	Descriptio n
	215	Status
	216	419696
	217	OR3.0431
	218	CPRS
	219	20080317
	220	Medium
	221	Error occu rred when user click ed on Prov ider Locat ion in CPR S.
	222	Resolved
	223	419712
	224	OR3.0431
	225	VistA
	226	20141210
	227	Medium
	228	Missing CP RS Pop-Up Box Flags.
	229	Resolved
	230	419795
	231	OR3.0431
	232	VistA
	233	20141210
	234	Medium
	235	Missing “I neligible” flag for CPRS Pop-U p Box func tionality.
	236	Resolved
	237	419989
	238	OR3.0431
	239	CPRS
	240	20141210
	241	High
	242	Missing Po p-Up Flags .
	243	Resolved
	244	419992
	245	OR3.0431
	246	VistA
	247	20141210
	248	Medium
	249	Code clean up needed for this N SR.
	250	Resolved
	251	419996
	252	OR3.0431
	253	VistA
	254	20141210
	255	Medium
	256	Installati on instruc tions for OR3.0431 need to b e edited.
	257	Resolved