S60 Smartphone Quality Assurance: A Guide for Mobile Engineers and Developers

About the Author  xiii

Preface xv

Chapter 1: Introduction to S60  1

1.1 The Competitive Advantage of the S60 Platform  3

1.2 S60 Architecture  5

1.2.1 The Symbian Operating System (Symbian OS)  6

1.2.2 Domestic Operating System (DOS)  6

1.2.3 User Interface (UICon)  7

1.3 Summary  7

Chapter 2: Selecting the Baseline  9

2.1 Manny Lehman’s Law  10

2.2 What is so Challenging about Selecting the Best Baseline?  11

2.3 How should the Baseline be Selected?  12

2.3.1 Baseline Maturity  13

2.3.2 Customization Maturity  13

2.3.3 Least Stable Sub-system  14

2.3.4 Program Timing  14

2.4 Summary  14

Chapter 3: Release Management  17

3.1 The Build Cycle  19

3.2 Required Testing Activities  23

3.3 Summary  23

Chapter 4: Binary Compatibility  25

4.1 API Categorization  28

4.2 Maintaining Compatibility  30

4.2.1 Platform Compatibility  30

4.2.2 Platform-based Phone Compatibility  31

4.2.3 Application Compatibility  32

4.2.4 Compatibility Dimensions  32

4.3 Binary Compatibility Scenario  33

4.4 Binary Compatibility Verifi cation  35

4.4.1 The Binary Compatibility Verifi cation Process  35

4.4.2 The Binary Compatibility Verifi cation Suite  36

4.4.2.1 The SDK Analyser  36

4.4.2.2 The Source Analyser  38

4.4.2.3 The Binary Analyser  39

4.4.2.4 The Application Launcher  39

4.4.2.5 Binary Compatibility Applications  40

4.4.2.6 Third-Party Applications  40

4.5 Possible Future Tools  40

4.5.1 DepInfo Tool  41

4.5.2 Header Checker Tool  41

4.5.3 Ordinal Checker  42

4.6 Summary  42

Chapter 5: Certifi cates and Standards  43

5.1 Technology Certifi cates  44

5.1.1 Java/TCK  44

5.1.2 Bluetooth  47

5.1.2.1 BT Certifi cation Areas  47

5.1.3 Other Technology Licences  48

5.1.4 Security Certifi cates  49

5.1.5 Universal Serial Bus  50

5.1.6 Infrared Connectivity  50

5.1.7 Multimedia Cards (MMC)  51

5.2 The Open Mobile Alliance (OMA)  52

5.2.1 Process and Principles  52

5.3 Cellular Standards and Operators  55

5.3.1 Government and Quality Certifi cates  56

5.3.1.1 Mandatory  56

5.3.2 Optional  58

5.4 Summary  61

Chapter 6: What Quality Means  63

6.1 Quality Culture  64

6.2 Quality Standards  66

6.2.1 ISO 9000  66

6.2.2 Six Sigma  67

6.3 Quality in a Product  68

6.3.1 Quality in Manufacturing  69

6.3.2 Quality in Service  70

6.3.3 Getting Better Quality  71

6.4 Quality in the S60 Platform and S60-based Phones  73

6.4.1 Choosing the Process  73

6.4.2 The Waterfall Process  73

6.4.3 The Incremental Process  74

6.4.4 Agile Software Development  75

6.4.5 Concurrent Engineering  76

6.4.6 Other Things to Consider  77

6.5 Summary  78

Chapter 7: Stumbling Blocks  79

7.1 Stumbling Blocks General to All Projects  79

7.2 Stumbling Blocks Specifi c to a Software Program  80

7.2.1 Contradictory, Overwhelming or Too Many Requirements  83

7.2.2 Unstable, Incomplete and Informal Requirements  83

7.2.3 Poor Planning and Project Management  84

7.2.4 Unrealistic Estimates and Unjustifi ed Expectations  84

7.2.5 Lack of Knowledge on New Technologies  84

7.2.6 Lack of Proper Risk Management  85

7.2.7 Lack of Organisational Integrity  86

7.3 Ways to Avoid Stumbling Blocks in a Software Program  87

7.3.1 Overview of COCOMO  87

7.4 Stumbling Blocks Specifi c to a S60-based Phone Program  88

7.4.1 Program-level Risks  89

7.4.1.1 Integration Competence  89

7.4.1.2 Testing Environment  90

7.4.1.3 Amount of Differentiation  90

7.4.1.4 Baseline Selection  91

7.4.1.5 Defect Fixing Order  91

7.4.2 Component-level Risks  91

7.4.2.1 Coding Style and Culture  92

7.4.3 Fixing Speed  94

7.4.3.1 Testing Activities and Extent  94

7.4.3.2 Insuffi ciency of the Specifi cation  96

7.4.3.3 Adaptation Layer Implementation  96

7.5 Provider Components  97

7.6 Summary  97

Chapter 8: Platform Testing versus Platform-based Phone Testing  99

8.1 The S60 Testing Process  100

8.1.1 Platform Test Execution Process  100

8.1.1.1 Module/Component Testing  102

8.1.1.2 Sub-system Integration Testing  102

8.1.1.3 Basic Acceptance Testing (BAT)  102

8.1.1.4 Functional Testing  103

8.1.1.5 System, Localization, Binary Compatibility and Interoperability Testing  103

8.1.1.6 Release Testing  104

8.1.1.7 Regression Testing  104

8.1.1.8 Maintenance Testing  104

8.1.1.9 S60-based Phone Testing  104

8.1.1.10 Planning based on Baseline Maturity Analysis  107

8.1.1.11 Planning based on Fix Analysis  108

8.2 Summary  108

Chapter 9: Testing as a Tool  109

9.1 Testing in Different Processes  111

9.1.1 Testing in an Iterative Process  113

9.1.2 Testing in an Incremental Process  114

9.1.3 Testing in an Agile Process  114

9.1.4 Testing in an Extreme Programming Process  115

9.2 Testing Techniques  115

9.3 Testing Phases  120

9.3.1 Documentation Testing  121

9.3.2 Module Testing  121

9.3.3 Integration Testing in the Small  124

9.3.4 Functional Testing  126

9.3.5 Non-functional Testing  126

9.3.6 Integration Testing in the Large  127

9.3.7 The Real User Experience (TRUE)  128

9.4 What then?  131

9.5 Summary  132

Chapter 10: The Testing Environment  133

10.1 Module Testing  134

10.2 Integration Testing in the Small  135

10.3 Functional Testing  135

10.3.1 Common  136

10.3.2 UI Customization and Personalization  137

10.3.3 Local Connectivity  138

10.3.4 Networking and Data Bearers  138

10.3.5 Telephony  139

10.3.6 Multimedia  140

10.3.7 Personal Information Management (PIM)  141

10.3.8 Messaging  141

10.3.9 Browsing  143

10.4 Performance Testing  144

10.5 Interoperability Testing  145

10.6 Miscellaneous Testing Activities  146

10.6.1 Certifi cation  147

10.6.2 Usability  147

10.7 Summary  148

Chapter 11: Defect Analysis  149

11.1 Focused Testing  152

11.2 Defect Analysis and Reporting  153

11.2.1 Defect Database  154

11.2.2 The Defect Management Process  154

11.2.3 Defect Priority  156

11.2.3.1 Show Stopper  156

11.2.3.2 Critical  158

11.2.3.3 Major  159

11.2.3.4 Minor  159

11.2.4 Defect Reporting  160

11.3 Summary  161

Chapter 12: Integration and Build Environment  163

12.1 Software Confi guration Management  163

12.2 Changing the Code  164

12.2.1 Confi guration Management  166

12.3 Build Environment  167

12.3.1 Delivery Structure  167

12.3.2 Build Process  168

12.3.3 Build Tools  169

12.4 S60 Integration  171

12.4.1 Stage 1  171

12.4.1.1 Step 1: Successful Boot to Textshell  171

12.4.1.2 Step 2: Simple Application and Launch via WSER  172

12.4.1.3 Step 3: Starter Integration and Calculator Launch  172

12.4.1.4 Step 4: Complete the S60 Boot  172

12.4.2 Stage 2  172

12.4.3 Stage 3  173

12.4.4 Stage 4  173

12.4.5 Stage 5  173

12.5 Summary  173

Appendix A: Examples of S60 Devices  175

Appendix B: Glossary  179

Appendix C: References  187

Chapter 4: Binary Compatibility  187

Chapter 5: Certifi cates and Standards  187

Chapter 6: What Quality Means  188

Chapter 7: Stumbling Blocks  189

Chapter 8: Platform Testing versus

Platform-based Phone Testing  189

Chapter 9: Testing as a Tool  189

Chapter 11: Defect Analysis  190

Appendix D: Further Reading  191

Index 193