Produced by Araxis Merge on 8/15/2018 12:48:16 PM Central Daylight Time. See www.araxis.com for information about Merge. This report uses XHTML and CSS2, and is best viewed with a modern standards-compliant browser. For optimum results when printing this report, use landscape orientation and enable printing of background images and colours in your browser.
# | Location | File | Last Modified |
---|---|---|---|
1 | EPIP_Combined_CiF.zip\EPIP_Combined_CiF\5.2 | EPIP_Test_Evaluation_(LR_5.2_508).doc | Tue Aug 14 14:40:22 2018 UTC |
2 | EPIP_Combined_CiF.zip\EPIP_Combined_CiF\5.2 | EPIP_Test_Evaluation_(LR_5.2_508).doc | Tue Aug 14 14:48:34 2018 UTC |
Description | Between Files 1 and 2 |
|
---|---|---|
Text Blocks | Lines | |
Unchanged | 3 | 550 |
Changed | 2 | 4 |
Inserted | 0 | 0 |
Removed | 0 | 0 |
Whitespace | |
---|---|
Character case | Differences in character case are significant |
Line endings | Differences in line endings (CR and LF characters) are ignored |
CR/LF characters | Not shown in the comparison detail |
No regular expressions were active.
1 | Existing P roduct Int ake Progra m (EPIP) | |
2 | Patch LR*5 .2*508 | |
3 | Test Evalu ation | |
4 | ||
5 | Department of Vetera ns Affairs | |
6 | August 201 8 | |
7 | Version 1. 0 | |
8 | Revision H istory | |
9 | Note: The revision h istory cyc le begins once chang es or enha ncements a re request ed after t he Communi cations Pl an has bee n baseline d. | |
10 | DateVersio nDescripti onAuthor08 /06/20181. 0Initial d ocumentEPI P Project TeamArtifa ct Rationa le | |
11 | The test e valuation document i s the prim ary output of the te st and eva luation pr ocess, an integral p art of the systems e ngineering process, which iden tifies lev els of per formance a nd assists the devel oper in co rrecting d eficiencie s. | |
12 | Table of C ontents | |
13 | 11. | |
14 | Test Evalu ation Intr oduction | |
15 | ||
16 | ||
17 | 11.1. | |
18 | Test Evalu ation Scop e | |
19 | ||
20 | ||
21 | 11.2. | |
22 | Test Archi tecture | |
23 | ||
24 | ||
25 | 11.3. | |
26 | Test Envir onment/Con figuration | |
27 | ||
28 | ||
29 | 21.4. | |
30 | Installati on Process | |
31 | ||
32 | ||
33 | 22. | |
34 | Test Data | |
35 | ||
36 | ||
37 | 23. | |
38 | Issues | |
39 | ||
40 | ||
41 | 24. | |
42 | Test Execu tion Log | |
43 | ||
44 | ||
45 | 35. | |
46 | Test Defec t Log | |
47 | ||
48 | ||
49 | 36. | |
50 | Test Resul ts Summary | |
51 | ||
52 | ||
53 | 36.1. | |
54 | Defect Sev erity and Priority L evels | |
55 | ||
56 | ||
57 | 36.2. | |
58 | Total Defe cts by Sev erity Leve l | |
59 | ||
60 | ||
61 | 46.3. | |
62 | Breakdown of Test Re sults | |
63 | ||
64 | ||
65 | 46.4. | |
66 | Performanc e Testing | |
67 | ||
68 | ||
69 | 47. | |
70 | Test Cover age | |
71 | ||
72 | ||
73 | 47.1. | |
74 | Requiremen ts Covered | |
75 | ||
76 | ||
77 | 47.2. | |
78 | Section 50 8 Complian ce Coverag e | |
79 | ||
80 | ||
81 | 48. | |
82 | Suggested Actions | |
83 | ||
84 | ||
85 | 49. | |
86 | Defect Sev erity and Priority D efinitions | |
87 | ||
88 | ||
89 | 59.1. | |
90 | Defect Sev erity Leve l | |
91 | ||
92 | ||
93 | 59.1.1. | |
94 | Severity L evel 1 – C ritical | |
95 | ||
96 | ||
97 | 59.1.2. | |
98 | Severity L evel 2 - H igh | |
99 | ||
100 | ||
101 | 69.1.3. | |
102 | Severity L evel 3 - M edium | |
103 | ||
104 | ||
105 | 69.1.4. | |
106 | Severity L evel 4 - L ow | |
107 | ||
108 | ||
109 | 69.2. | |
110 | Priority C lassificat ions | |
111 | ||
112 | ||
113 | 69.2.1. | |
114 | Priority 1 - Resolve Immediate ly | |
115 | ||
116 | ||
117 | 69.2.2. | |
118 | Priority 2 - Give Hi gh Attenti on | |
119 | ||
120 | ||
121 | 69.2.3. | |
122 | Priority 3 - Normal Queue | |
123 | ||
124 | ||
125 | 69.2.4. | |
126 | Priority 4 - Low Pri ority | |
127 | ||
128 | ||
129 | 710. | |
130 | Optional T ables, Cha rts, and G raphs | |
131 | ||
132 | ||
133 | 811. | |
134 | Document A pproval Si gnatures | |
135 | ||
136 | ||
137 | 9Appendix A - Test E xecution L og | |
138 | ||
139 | ||
140 | 10Appendix B – Defec t Log | |
141 | ||
142 | ||
143 | ||
144 | ||
145 | Test Evalu ation Intr oduction | |
146 | The purpos e of this Test Evalu ation is t o: | |
147 | Identify t he testing approach used. | |
148 | Present a summary an alysis of the key te st results from the remediatio n of this intake for review an d assessme nt by desi gnated sta keholders. | |
149 | Provide a general st atement of the quali ty of the system und er test. | |
150 | Make recom mendations for futur e testing efforts. | |
151 | Test Evalu ation Scop e | |
152 | The scope of this Te st Evaluat ion is to verify the functiona lity of th e Patch LR *5.2*508 c ode modifi cation, as determine d by Funct ional, Com ponent Int egration/S ystem, and Regressio n testing. Testing a ctivities followed t he specifi cations ou tlined in the follow ing Master Test Plan : LR*5.2*5 08 Master Test Plan (included in Appendi x A). | |
153 | Test Archi tecture | |
154 | Following are the EP IP test ac counts use d by the L eidos Deve lopment an d SQA Test ing teams to test LR *5.2*508. | |
155 | Developmen t Test Acc ounts | |
156 | (For Unit Testing)SQ A Test Acc ounts | |
157 | (For Funct ional, Reg ression, a nd Compone nt Integra tion and S ystem Test ing)VistAS 1 (alterna te name: D 1S1)VistAG 1 (alterna te name: D 1G1) VistA S2 (altern ate name: D1S2)VistA G2 (altern ate name: D1G2) – | |
158 | for CPRS G UI testing onlyTest Environmen t/Configur ation | |
159 | The EPIP t est accoun ts are mai ntained by the EPIP System Adm inistrator , who inst alls all V A-released patches a s soon as they are n ationally released. All EPIP t est accoun ts are clo ned from e xisting VA Enterpris e Testing Services ( ETS) test accounts. The Comput erized Pat ient Recor d System ( CPRS) Grap hical User Interface (GUI) exe cutable is configure d for each VistA ins tance util izing a un ique Inter net Protoc ol (IP) ad dress to c onnect to the VistA applicatio ns. Any up dates to t he CPRS GU I executab le are han dled by th e EPIP Sys tem Admini strator. | |
160 | All EPIP T est Engine ers and De velopers w ho have th e proper c redentials can acces s the test accounts. The VA Au stin Infor mation Tec hnology Ce nter (AITC ) support team reset s password s and sets up new ac cess crede ntials on an as-need ed basis. | |
161 | Installati on Process | |
162 | As soon as the remed iation pro cess is co mplete and the patch is availa ble for te sting, a K IDS build is created in the De velopment account an d then sen t to FORUM for final packaging . The patc h is then submitted to the VA SQA Lead’s Mailman a ccount for installat ion. | |
163 | An EPIP De veloper or Test Engi neer utili zes the KI DS Install ation proc ess to ext ract the b uild from the patch and instal l the buil d into a t est accoun t. The ind ividual wh o installs the patch verifies the routin e checksum s and also checks fo r errors d uring the installati on process . If the p atch is su ccessfully installed without a ny errors, then the EPIP Test team proce eds with F unctional, Regressio n, and Com ponent Int egration a nd System testing. I f defects are found, then the Developmen t team wor ks to find a resolut ion and cr eates new versions o f the patc h until al l defects are resolv ed. | |
164 | Test Data | |
165 | The SQA Te sting team utilizes the test d ata in the designate d test acc ounts (D1G 1, D1G2). | |
166 | The test d ata is enc rypted fol lowing the standards set forth by the VA Office of Informati on & Techn ology (OIT ). All Per sonally Id entifiable Informati on (PII) a nd Protect ed Health Informatio n (PHI) is scrubbed and is not available to the Te st Enginee rs. | |
167 | All testin g is execu ted using encrypted test patie nts availa ble from a ny of the EPIP test accounts. Examples o f encrypte d test pat ients: | |
168 | AAAHURMMX, XPHY | |
169 | BADHB, HAA DXS | |
170 | FDHUX, YHI J | |
171 | All tests were execu ted manual ly by EPIP Test Engi neers. | |
172 | Issues | |
173 | No issues were encou ntered dur ing testin g of LR*5. 2*508. | |
174 | TitleIssue Descripti onTypeSeve rityN/AN/A N/AN/ATest Execution Log | |
175 | The Test E xecution L og records the execu tion of te st scripts and docum ents the t est result s for each test scri pt. | |
176 | The SQA Te sting team utilizes the Ration al Quality Manager ( QM) tool f or all tes ting activ ities. All test docu ments are stored in the EPIP r epository, including the Maste r Test Pla n, Test Su ites, Test Cases, an d Test Scr ipts. Test execution is perfor med, and t est result s recorded , in QM. T he Test En gineer add s the test results t o the Test Execution records t o indicate whether t esting ach ieved Pass or Fail s tatus. | |
177 | The Test E xecution r ecords for LR*5.2*50 8 are incl uded in th e EPIP Pat ch LR*5.2* 508 Master Test Plan . The Mast er Test Pl an is avai lable in A ppendix A. | |
178 | Test Defec t Log | |
179 | The Test D efect Log is a tool for record ing, analy zing, trac king, and documentin g the clos ure of def ects. It s pecifies t he screen, field, be havior or result tha t occurred , and the IEEE-defin ed Severit y Level. I t includes enough in formation for the de veloper to find and re-create the defect . The Defe ct Log is available in Appendi x B. | |
180 | Test Resul ts Summary | |
181 | This patch was devel oped to re move one o bsolete fi eld refere nce for My cology and update on e specimen for Bacte riology in the EXECU TE CODE fi le (#62.07 ). The obs olete refe rences wer e discover ed after t he release of LR*5.2 *476. | |
182 | SQA testin g for this patch sta rted on Ma y 22, 2018 and ended on July 3 1, 2018. | |
183 | Test versi on 1 was i nstalled i n the Dev1 Gold1 tes t environm ent after Unit testi ng in Dev1 Silver1 w as complet ed. Upon c ompletion of Integra tion testi ng (Compon ent Integr ation and System Tes ting, Func tional Tes ting, and Regression Testing), zero (0) defects we re found a nd reporte d. However , HPS requ ested an u pdate afte r testing was comple te. The up date adds new DESCRI PTION fiel d (#6) tex t in the E XECUTE COD E file (#6 2.07). | |
184 | Test versi on 2 was c reated to address th e request made by HP S. The pat ch was ins talled in the Dev1 G old1 test environmen t on June 6, 2018 af ter Unit t esting in Dev1 Silve r1 was com pleted. Up on complet ion of Fun ctional an d Regressi on testing on June 2 0, 2018, z ero (0) de fects were found and reported. However, HPS reques ted an upd ate after testing wa s complete . The upda te enables predefine d prompt t ext from t he LAB DES CRIPTIONS file (#62. 5) to appe ar when th e user run s the Resu lts Entry (Batch) op tion. | |
185 | Test versi on 3 was c reated to address th e request made by HP S. The pat ch was ins talled in the Dev1 G old1 test environmen t on June 29, 2018 a fter Unit testing in Dev1 Silv er1 was co mpleted. U pon comple tion of Fu nctional a nd Regress ion testin g on July 12, 2018, one (1) de fect was f ound and r eported. | |
186 | Test versi on 4 was i nstalled i n the Dev1 Gold1 tes t environm ent on Jul y 27, 2018 after Uni t Testing in Dev 1 S ilver1 was completed . Upon com pletion of Functiona l and Regr ession tes ting, zero (0) defec ts were fo und and re ported. | |
187 | Defect Sev erity and Priority L evels | |
188 | A defect i s defined as a flaw in a compo nent or sy stem that can cause the compon ent or sys tem to fai l to perfo rm its req uired func tion, e.g. , an incor rect state ment or da ta definit ion. A def ect, if en countered during exe cution, ma y cause a failure of the compo nent or sy stem. | |
189 | Defects ar e categori zed accord ing to sev erity and priority l evels. The test anal yst assign s the seve rity, whil e the deve lopment ma nager assi gns the pr iority for repair. F or more in formation, see Defec t Severity and Prior ity Defini tion in th is Test Ev aluation. | |
190 | Total Defe cts by Sev erity Leve l | |
191 | The Defect Log in Ap pendix B d isplays th e defects encountere d while te sting this patch, an d the seve rity level of each. | |
192 | Breakdown of Test Re sults | |
193 | Testing wa s complete d on July 31, 2018. All test r esults wer e recorded in QM. De tailed res ults are a vailable i n the EPIP Patch LR* 5.2*508 Ma ster Test Plan (see Appendix A ). | |
194 | Performanc e Testing | |
195 | Performanc e testing was not co nducted. | |
196 | Test Cover age | |
197 | The EPIP P atch LR*5. 2*508 Mast er Test Pl an contain s details on test co verage (se e | |
198 | Appendix A ). | |
199 | Requiremen ts Covered | |
200 | Patch LR*5 .2*508 res ulted from HPS defec t 742951. The defect is stored in the Ra tional Cha nge and Co nfiguratio n Manageme nt (CM) mo dule. The test case used to va lidate def ect resolu tion is st ored in th e Quality Manager (Q M) module. Both modu les are lo cated with in the EPI P namespac e. | |
201 | The follow ing links provide ac cess to th e EPIP CM and QM mod ules withi n Rational : | |
202 | EPIP (CM) – Go to Pl ans, then All Plans, and then search for the desir ed sprint Plan. The user stori es in each Plan are linked to requiremen ts and tes t cases. | |
203 | https://cl m.rational .oit. DNS /ccm/web/p rojects/EP IP%20(CM)# action=com .ibm.team. dashboard. viewDashbo ard | |
204 | EPIP (QM) – Go to Pl anning, th en Browse Test Plans , and then search fo r the desi red Master Test Plan . The Mast er Test Pl an and tes t cases ar e linked t o requirem ents. | |
205 | https://cl m.rational .oit. DNS /qm/web/co nsole/EPIP %20(QM)#ac tion=com.i bm.rqm.pla nning.home .actionDis patcher&su bAction=vi ewUserHome | |
206 | Section 50 8 Complian ce Coverag e | |
207 | Section 50 8 testing was not re quired for this patc h. This wi ll be repo rted to VA by submit tal of the following document: | |
208 | EPIP_VASec tion508_In take_Docum ent_(LR*5. 2*508).pdf | |
209 | Suggested Actions | |
210 | Leidos rec ommends mo ving this patch to I OC testing . | |
211 | Defect Sev erity and Priority D efinitions | |
212 | The classi fication o f defects within a s ystem exam ines both the severi ty and pri ority of t he defect. | |
213 | Severity i s a measur e of how g reat the i mpact is o n the user ’s ability to comple te the doc umented ac tions with in the sys tem. | |
214 | Priority d etermines the speed with which a given d efect must be repair ed. | |
215 | Defect cla ssificatio n may be d etermined either bec ause testi ng is dela yed by a f ailure in the system or becaus e a cumber some worka round prev ents a use r from com pleting th e assigned tasks. Bo th severit y and prio rity measu res must b e recorded when sche duling def ect resolu tion tasks . | |
216 | Defect Sev erity Leve l | |
217 | The follow ing subsec tions iden tify the d efect seve rity level s. | |
218 | Severity L evel 1 – C ritical | |
219 | Institute of Electri cal and El ectronics Engineers (IEEE) def inition: T he defect results in the failu re of the complete s oftware sy stem, of a subsystem , or of a software u nit (progr am or modu le) within the syste m. | |
220 | Any defect that comp romises pa tient safe ty or syst em securit y. Example s of syste m security defects i nclude bre ach of con fidentiali ty require ments of t he Privacy Act, the Health Ins urance Por tability a nd Account ability Ac t (HIPAA), or Federa l Tax Info rmation gu idelines. | |
221 | Loss of sy stem funct ionality c ritical to user oper ations wit h no suita ble workar ound, i.e. , there is no way to achieve t he expecte d results using the applicatio n. | |
222 | System cra sh or hang that prev ents furth er testing or operat ion of the complete applicatio n or a sec tion of th e applicat ion. | |
223 | Any defect that caus es corrupt ion of dat a from a r esult of t he system (as oppose d to user error). | |
224 | Any defect in which inappropri ate transm issions ar e consiste ntly gener ated or ap propriate transmissi ons of HL7 messages fail to be generated . | |
225 | Loss of fu nctionalit y resultin g in erron eous eligi bility/enr ollment de terminatio ns or comm unications not being sent. | |
226 | Severity L evel 2 - H igh | |
227 | IEEE defin ition: The defect re sults in t he failure of the co mplete sof tware syst em, of a s ubsystem, or of a so ftware uni t (program or module ) within t he system. There is no way to make the f ailed comp onent(s) f unction. H owever, th ere are ac ceptable p rocessing alternativ es which w ill yield the desire d result. | |
228 | A major de fect in th e function ality that does not result in corruption of data. | |
229 | A major de fect in th e function ality resu lting in a failure o f all or p art of the applicati on, where: | |
230 | The expect ed results can tempo rarily be achieved b y alternat e means. T he custome r indicate s the work around is acceptabl e for the short term . | |
231 | Any defect that does not confo rm to Sect ion 508 st andards. | |
232 | Any defect that resu lts in ina ccurate or missing r equirement s. | |
233 | Any defect that resu lts in inv alid authe ntication or authent ication of an invali d end user . | |
234 | Severity L evel 3 - M edium | |
235 | IEEE defin ition: The defect do es not res ult in a f ailure, bu t causes t he system to produce incorrect , incomple te, or inc onsistent results, o r the defe ct impairs the syste ms usabili ty. | |
236 | Minor func tionality is not wor king as in tended and a workaro und exists but is no t suitable for long term use | |
237 | The inabil ity of a v alid user to access the system consisten t with gra nted privi leges | |
238 | Typographi cal or gra mmatical e rrors in t he applica tion, incl uding inst allation g uides, use r guides, training m anuals, an d design d ocuments | |
239 | Any defect producing cryptic, incorrect, or inappr opriate er ror messag es | |
240 | Any defect that resu lts from t he use of non-standa rd data te rminology in the app lication o r document ation, as defined by the Depar tment of V eterans Af fairs | |
241 | Cosmetic i ssues that are impor tant to th e integrit y of the p roduct, bu t do not r esult in d ata entry and or dat a quality problems. | |
242 | Severity L evel 4 - L ow | |
243 | IEEE defin ition: The defect do es not cau se a failu re, does n ot impair usability, and the d esired pro cessing re sults are easily obt ained by w orking aro und the de fect. | |
244 | Minor loss of, or de fect in th e function ality wher e a long t erm use ex ists | |
245 | Low-level cosmetic i ssues. | |
246 | Priority C lassificat ions | |
247 | The follow ing subsec tions iden tify the a ppropriate actions f or defects at each p riority le vel, per d efinitions of IEEE. | |
248 | Priority 1 - Resolve Immediate ly | |
249 | Further de velopment and/or tes ting canno t occur un til the de fect has b een repair ed. The sy stem canno t be used until the repair has been affe cted. | |
250 | Priority 2 - Give Hi gh Attenti on | |
251 | The defect must be r esolved as soon as p ossible be cause it i s impairin g developm ent and/or testing a ctivities. System us e will be severely a ffected un til the de fect is fi xed. | |
252 | Priority 3 - Normal Queue | |
253 | The defect should be resolved in the nor mal course of develo pment acti vities. It can wait until a ne w build or version i s created. | |
254 | Priority 4 - Low Pri ority | |
255 | The defect is an irr itant that should be repaired, but can b e repaired after mor e serious defects ha ve been fi xed. | |
256 | Optional T ables, Cha rts, and G raphs | |
257 | None. | |
258 | Document A pproval Si gnatures | |
259 | Signed: __ __________ __________ __________ __________ __________ __________ _________ | |
260 | Program/Pr oject Mana ger | |
261 | Date | |
262 | Signed: __ __________ __________ __________ __________ __________ __________ _________ | |
263 | Business S ponsor Rep resentativ e | |
264 | Date | |
265 | Signed: __ __________ __________ __________ __________ __________ __________ _________ | |
266 | Test Lead | |
267 | Date | |
268 | Appendix A - Test Ex ecution Lo g | |
269 | The Test E xecution R ecords for LR*5.2*50 8 are incl uded in th e EPIP Pat ch LR*5.2* 508 Master Test Plan . | |
270 | ||
271 | Appendix B – Defect Log | |
272 | This patch was devel oped to ad dress two (2) minor data error s discover ed during the releas e of patch LR*5.2*47 6. The err ors resolv ed by patc h LR*5.2*5 08 are des cribed bel ow. The HP S defect n umber is 7 42951. | |
273 | In the EXE CUTE CODE entry for BACTERIOLO GY, there is an inco rrect Inte rnal Entry Number (I EN) for Sp utum. This patch rep laces IEN 68 with IE N 360. | |
274 | In the EXE CUTE CODE entry for MYCOLOGY, there is a reference to field 19.2. This field is not define d in the L AB DATA fi le (#63) a nd was rem oved. | |
275 | Three (3) SQA defect s were ent ered durin g testing of version s 1 throug h 4. All d efects hav e been res olved. | |
276 | SQA Defect IDAffecte d ScreenAf fected Fie ldObserved BehaviorS everityDes criptionN/ AN/AN/AN/A N/ANo defe cts were f ound durin g Unit Tes ting of ve rsion 1.0. N/AN/AN/AN /AN/ANo de fects were found dur ing Compon ent Integr ation/Syst ems Testin g of versi on 1.0.N/A N/AN/AN/AN /ANo defec ts were fo und during Regressio n testing of version 1.0.76335 1FileManDE SCRIPTION field (#6) in the EX ECUTE CODE file (#62 .07)N/ALow No defects were foun d during F unctional Testing of version 1 .0. Howeve r, HPS sub sequently requested field text updates, requiring a new buil d. SQA def ect 763351 was creat ed to addr ess this r equest. N/ AN/AN/AN/A N/ANo defe cts were f ound durin g Unit Tes ting of ve rsion 2.0. N/AN/AN/AN /AN/ANo de fects were found dur ing Regres sion testi ng of vers ion 2.0.78 8417VistAP reliminary Comments for Microb iology, My cology and TB Bacter iologyUnab le to expa nd lab des criptionsH ighNo defe cts were f ound durin g Function al Testing of versio n 2.0. How ever, HPS subsequent ly request ed changes to predef ined promp t text, re quiring a new build. SQA defec t 788417 w as created to addres s this req uest. N/AN /AN/AN/AN/ ANo defect s were fou nd during Unit Testi ng of vers ion 3.0.N/ AN/AN/AN/A N/ANo defe cts were f ound durin g Regressi on testing of versio n 3.0.7884 07Laborato ry DHCP Me nu > Micro biology Me nu > Resul ts Entry ( batch)Prel iminary TB CommentUn able to ex pand lab d escription MediumOne (1) defect was found during Fu nctional T esting of version 3. 0. N/AN/AN /AN/AN/ANo defects w ere found during Uni t Testing of version 4.0.N/AN/ AN/AN/AN/A No defects were foun d during R egression testing of version 4 .0.N/AN/AN /AN/AN/ANo defects w ere found during Fun ctional te sting of v ersion 4.0 . | |
277 | _159574476 4.pdf |
Araxis Merge (but not the data content of this report) is Copyright © 1993-2016 Araxis Ltd (www.araxis.com). All rights reserved.