7:30-8:00 AM

Registration and Exhibits open

8:00-8:15 am

Welcome and business announcements

8:15-9:45 am

Keynote address: The Art of Building Consensus, Sam Kaner

9:45-10:15 am

Break in Exhibit Hall

COLLABORATION

COLLABORATION

TESTING

ASSORTMENT

10:15 - 11:00 AM

INVITED SPEAKERS

Expanding Trust and Collaboration with Earned Value Tracking

Tamara Sulaiman

Actually, It IS All About You!,

Jim Brosseau, [P-6]

Testing for the User Experience,

Lanette Creamer, [P-24]

Quality Management of Outsourced Projects,

Ying Kwong, [P-46]

11:10 - 11:55 AM

“Ideastreaming”: Harnessing the Power of Collaborative Innovation Software,

John Simpson, [P-22]

Software Testing as a Service (STaaS),

Leo Van der Aalst, [P-8]

Building a Successful Multi-Site Team Using Chocolate,

Doug Whitney, [P-17]

1:35-2:20 PM

Collaboration - Modern Techniques for Quality Initiatives,

Jeff Fiebrich [P-25]

PANEL

Collaboration - Learning from Other Industries

Is Your Testing

Effective and Efficient?,

Bhushan Gupta [P-9]

Distributed Agile:

An Experience Report,

Joy Shafer [P-37]

2:30-3:15 PM

Trust: The Key to Project Team Collaboration,

Diana Larsen, [P-39]

TBA

Acceptance Testing: A Love Story in Two Acts,

Jon Bach, [P-16]

Case Study: Fostering Meaningful Change with the Large Format

Printer Division at HP,

Jim Brosseau, [P-7]

4-4:45 PM

INVITED SPEAKER

Playing Nice in the Sandbox,

Janet Gregory

Collaborative Quality: One Company’s Recipe for Software Success,

Alan Ark, [P-12]

Building a Software Testing Strategy,

Karen Johnson, [P-49]

Collaboration Among Software Components,

Dick Hamlet, [P-14]

4:55-5:40 PM

Tails in the Boardroom: Canine Lessons for Business Teams,

Shannon MacFarlane , [P-4]

Test Case Generation Design Pattern,

Lian Yang, [P-50]

Selecting Software Estimating Techniques that Fit the Software Process,

Kal Toth, [P-41]

7:30-8:00 AM

Registration open

8:00-9:30 am

Keynote Address: Quality Dynamics of Agile Software Development, Ron Jeffries & Chet Hendrickson

9:30 -10:00 am

Break with limited Exhibits & Poster Papers

COLLABORATION

MANAGEMENT

TESTING

ASSORTMENT

10:00-10:45 AM

Collaborative Techniques for the Determination of a Best Alternative in a Software Quality Environment,

James McCaffrey, [P-10]

INVITED SPEAKER

It’s Not Just an Update:

Using Status Reporting to

Expand Collaboration,

Mike Kelly

Non-Regression Test Automation,

Douglas Hoffman, [P-19]

Outnumbered: Ensuring Quality with a Low Test-to-Dev Ratio,

Brian Rogers, [P-29]

10:55 -11:40 AM

Quality Software Engineering: Collaboration Makes the Experience,

Diana Dukart, [P-18]

Web UI Automation - A Browser Agnostic Framework for Web UI Testing,

Manuel Merino [P-35]

The TAO of Software Defect Test Collaboration and Estimation,

James Eisenhauer, [P-11]

11:50 - 1:05 PM

1:20-2:05 PM

Collaborative Change,

Debra Lavell, [P-26]

Getting and Keeping Talent: Women in Software Development,

Sharon Buckmaster, [P-36]

TBA

Virtual Lab Management: Best Practices and Common Pitfalls,

Ian Knox, [P-27]

2:15-3:00 PM

Agile Retrospectives: Collaboration for Continuous Improvement,

Diana Larsen, [P-40]

Storytelling Techniques:

Reporting Product Status in a Meaningful Way,

Karen Johnson, [P-48]

Dealing with the Most

Influential Factors

Cause Customer Dissatisfaction

Duvan Luong

Vendor Presentation:

Web Application Security,

Ravi Yerra,

Arsin Corporation

3:30 - 4:15 PM

Quick Collaboration Between Theorists - Analyze This!,

Ian Savage, [P-44]

INVITED SPEAKER

Selling Your Ideas

to Management,

Steve Smith

What PNSQC Expects in a Paper,

Program Committee

Ensuring Software Quality for Large Maintenance Releases,

Jean Hartmann, [P-21]

4:25 5:10 PM

IT Collaboration at Stanford University,

Claudia Dencker, [P-20]

TBA

What I Learned at this Conference,

1-minute talks

Duvan Luong, Cadence Design Systems

For most of the successful companies, effectively addressing the key factors of customer dissatisfaction and the high cost of rework are the top-most items in the company business operational requirements list. It has been widely known that product defects have a very high correlation with the above two business success (or un-success) factors. Effectively handling product defects has a major impact on customer satisfaction and the eventual high cost of rework.

Product defects vary widely in impact to customer satisfaction and the cost of rework. Product crashes have a relative small distribution value in comparing with the total product defects (~ 20%) — however they cause an influential impact on customer satisfaction. Most product crashes are not fatal to the operation of neither the applications nor the business activities that use the applications. However, product crashes are annoying to customers, and many require resolution before operations can proceed. This paper describes the story of an organization who decided to address product crash issues.

Dr. Duvan Luong is as Engineer Director/Architect at Cadence Design Systems. Dr. Luong’s background is in the software development process and best practices with emphasis in testing. Other interests include the areas of organizational change, process improvement, customer satisfaction, effective consulting, problem solving, and operational excellence including metrics and statistical controls. Dr Luong has 29 years of experience in the Software Development Industry with AT&T and Bells Labs, IBM, Synopsys, Sun Microsystems, Hewlett Packard, and Cadence Design Systems.

Shannon MacFarlane, Totem Ocean Trailer Express

Dogs know the secrets to productivity and harmony within their packs; humans are often blind to these simple truths. Pack theory and canine behaviorism can be successfully applied to manage and improve human relationships. This study is an examination of how common canine behaviors within packs (e.g., conflict management, leadership, and communication) are equally effective when employed by members of business teams. Canine and human examples from current literature and observation illustrate the potential for improving efficiency, productivity, and collaboration among teams without raising hackles.

Shannon is the Quality Specialist at Totem Ocean Trailer Express, Inc., in Tacoma, Washington. She has the distinct honor of being TOTE’s only full-time quality employee. Her duties include caring for the health and wellness of the quality management system and directing document control, internal audit, corrective and preventive action, and TOTE’s best practice program. Shannon is a three-year Senior Examiner with the Washington State Quality Award and a first-year Baldridge Examiner. She has lots of experience squeezing productivity and good behavior from unruly team members and dogs.

Jim Brosseau, Clarrus Consulting Group Inc.

Even though technology projects require close collaboration and creativity to solve problems, there remains a strong tendency to apply industrial management techniques and work as individuals. The cost of these behaviors is not as clear as it can be in other fields, but the magnitudes can be devastating. If we really want to improve the way we develop technical solutions, we need to start by understanding the team itself - what each person bring to the table. The context of skills, cultures, and attitudes of all the people involved plays a critical role in determining which approach will be most effective, and indeed how we should deploy changes to our approach within the team.

While emphasis is generally placed on adoption of the hard skills such as new tools, techniques or methodologies, this personal and team context dramatically influences the overall results, and should be consciously managed.

After briefly describing the pitfalls of typical approaches to improvement, this presentation presents a sequence to follow for effective team change. Specific practices for consciously building technical teams are illustrated with case studies and anecdotes where soft-skills effectively solved difficult team issues. The need for effective follow through for change is discussed, and the author’s actual data from teams illustrates the need to think of what we generally call ‘best practices’ more as tools for effective collaboration among the team.

Jim Brosseau has a career spanning more than 20 years in a variety of roles and responsibilities. He has held successively more responsible positions in military and defense contracting, commercial software development, and training and consulting. He has worked in a QA role, acted as team lead, project manager, and director. In addition, Jim has worked with more than 80 organizations in the past 10 years with a goal of increasing collaborative effectiveness. An integral part of this effort has been a focus on techniques for measurement of productivity gains and ROI for refined development and management practices. His first book, Software Teamwork: Taking Ownership for Success, was published in 2007 by Addison-Wesley Professional.

Jim Brosseau
Carolina Altafulla, HP

Sustaining meaningful change in any organization can be difficult, particularly when there are strong personalities and established practices in place. Most initiatives are either too broad in their changes, or fail to address the needs of participants.

Within the Large Format Printer Division at Hewlett-Packard in Barcelona, despite some of the strongest front-end analysis practices in the industry, projects continued to face delivery challenges similar to those in many other companies.

This case study presents from two perspectives: inside the business and outside, from the external consultant. Both perspectives help us understand the complete picture, and we will see that only when the perspectives are brought together in a team environment do we see the complete value of the engagement.

Jim Brosseau, see P-6 for bio.

Carolina Altafulla has been in the technology industry since 1992, first in medical devices and later in the printing industry. She has worked in new product development as development engineer. In QA, she has acted as team lead and initiative manager for international and multidisciplinary projects. Carolina speaks fluently 6 languages. Carolina recently relocated with HP to the USA to broader her experience in Program management for new product development in the printing industry.

Leo Van der Aalst, Sogeti Netherlands B. V.

In analogy to Software as a Service (SaaS), I think Software Testing as a Service (STaaS) will be a next step in the field of test services. Why?

The drivers for SaaS adoption are also applicable to STaaS. The traditional rationale for outsourcing of IT systems is that by applying economies of scale to the operation of applications, a service provider can offer better, cheaper, more reliable applications than companies can themselves. The use of SaaS-based applications has grown dramatically, as reported by many of the analyst firms that cover the sector.

In the presentation, the presenter will share his perspective of the STaaS variants of the SaaS drivers, talk about the advantages of STaaS, and explain how STaaS could look like in practice and why it will work. For now; think of it as large virtual (digital) cupboard where clients put in their testing jobs, brokers distributing these jobs over the available testers and returning the results to the clients (all over the internet).

After having gone through the classic path (from programmer to project manager) Leo has specialized since 1990 in the test field, in which he has been a test manager, test advisor, research & development manager and line manager. As a business development manager, he co-authored TMap® Next and provides implementation services and outsourcing to organizations. Leo designed the EXIN tasks for TMap Next Foundation and for the Advanced exam and he supplies preparation training for these exams. His knowledge and experience in the project and test management field have been used and developed further during a number of international projects and consultancy trajectories (in USA, Germany, Denmark, and Austria). Leo is a sought-after teacher of test training and a regular speaker at national and international conferences (a.o. NGI, Test Congress in London, Quality Week Europe in Brussels, ICSTEST in Düsseldorf, TAV in Stuttgart). He is the author of several articles on risk-based testing, benefits of a test service center, and business-driven test management.

Bhushan Gupta, HP

An organization relies on its QA group to qualify its products. An adequate testing can be achieved relatively easily if your product is simple and does not involve multiple facets such as software, hardware, OEM components, and industry compliances including safety and environment. A product with multiple facets has increased product qualification complexity as multiple test teams get involved. It also becomes increasingly important to assure that all facets of the product are tested and there is no unintended test efforts duplication.

The RPS group at Hewlett-Packard has developed a technique that assures a high level of test effectiveness and efficiency and yields a high quality product. The technique involves identifying the quality attributes (test types) that must be tested for a product. These test types include Functionality, Usability, Reliability, Installation/Deployment, Safety, and Regulatory and form the horizontal test vector. To make sure that the product components are adequately tested before they are integrated, a vertical test vector representing development stages including Unit, Module, Component, System, and Solution and Beyond is also established. The two vectors together yield a highly effective and efficient “Test Landscape” that is used to determine test ownership and track test status from the very early stage in the product development lifecycle to the final release. The technique is being used by the group and has made test management simple yet efficient. It has also assured the management teams that the testing is effective and efficient. The Test Landscape has become a part of the Product Development Lifecycle and is reviewed at various checkpoints during the development. The group has also developed a roles and responsibility model that assures that the Test Landscape is properly executed to yield the maximum benefit.

Bhushan Gupta has 23 years of experience in software engineering, 13 of which have been in the software industry. Currently a Test Lead in RPS, Hewlett-Packard, he joined the company as a software quality engineer in 1997 where he was responsible for identifying key process areas to reduce rework. From 2003 to 2007 Bhushan was a Software Process Architect in the Indigo Division leading agile software development, customizing lifecycles, and measuring and improving software productivity. In his current position he program manages the new product qualification for the RPS group. From 1995-97 Bhushan worked as a Systems Analyst at Consolidated Freightways and before that he was a faculty member of the Software Engineering Technology Department at the Oregon Institute of Technology for 10 years.

Bhushan has served the PNSQC as a vice president, board member, and a committee chair. He has presented at several conferences including PNSQC and actively presents at special interest groups. He also presents a 2-day workshop at the OGI titled “Engineering Software Quality.”

James McCaffrey, VTE/Microsoft

This paper describes five techniques to collaboratively determine a single, best option from a list of alternatives in a software development environment. Techniques discussed include the pure plurality technique, the majority runoff technique, the Borda count system, the Condorcet method, and the Schulze method. We suggest that some of the key factors you should consider when using group evaluation techniques include the number of options, the number of evaluators, whether the alternatives are policy decisions or product decisions, and the extent to which evaluators are affected by the final result of the analysis.

Dr. James McCaffrey oversees technical training for software engineers working at Microsoft’s Redmond, Washington campus. James worked on several key Microsoft products including Internet Explorer, and is a Contributing Editor for Microsoft’s MSDN Magazine.

James Eisenhauer, Regence Blue Cross Blue Shield
Scott Martin, Regence Blue Cross Blue Shield

When millions of project dollars are at stake, estimating requires the focused efforts of its end users (stakeholders) and a great many subject matter experts (SMEs).

This paper will explore the collaborative process as it unfolded for two Regence analysts who were tasked with building a model that estimates the date of completion for the testing cycle, which included improving the software to release quality. Neither analyst had a clear view of the path forward, but through collaboration began to define the problem, identify the limitations (of both the techniques employed and the data available), and to construct a defendable model that the senior leadership team could confidently present to its board of directors. The paper chronicles many of the problems encountered, and limitations of implementing ideal solutions along with the authors experienced-based recommendations for readers facing similar challenges in their own profession.

Much has been written on the subject of software development and testing by noteworthy authors; it is not the intent of this paper’s authors to expound or refute the validity of their work. Rather, the intent is to enable the reader to identify the necessary stakeholders, explain the limitations inherent in all models and initiate a collaborative effort toward building an estimating model that works for their organization (within known constraints).

Scott D. Martin currently works at Regence Blue Cross Blue Shield of Oregon as a Performance Analyst for the Regence Information Technology Services division. Scott has over 16 years of multi-level collaborative experience across a broad range of industries and employers which includes two Fortune 100 companies (Raytheon and Hewlett Packard), and two large international companies (Japanese-based Kyocera and German-based Infineon Technologies). He has won repeated recognition for his work constructing a variety of forecasting and trending models, mapping process workflows and quantifying product contribution margins, improving supplier quality and enhancing corporate workforce performance through orchestrating broad behavior-based performance reforms. Scott earned an MBA from Portland State University, a BS in Business from the University of Oregon and a BA in Management from George Fox University.

James R. Eisenhauer is currently at Regence Blue Cross Blue Shield as a Software Process and Quality Analyst. Prior to Regence, James was a Sr. Software Architect at Lockheed Martin Information Technology, and was the driving force behind a progressive software solution approach to IT Service Delivery and Governance. During his tenure at Lockheed Martin, James was the lead consultant on many application architecture and software process engagements with major clients that included; Nike Inc, Goodyear Tire & Rubber, Symetra Financial, Department of Homeland Security, and the United Negro College Fund. He holds a Master’s of Business Administration (MBA) from the University of Tampa, Six Sigma Green Belt, ITIL Certification, ISO Auditor and a certificate of Software Engineering from the Oregon Graduate Institute School of Science and Engineering.

Alan Ark, Compli

Building software is not an easy process. There are many obstacles that can get in the way of delivering a great product in a timely manner. The intent of this paper is to share one company’s experiences with getting better software out the door. This paper does not give you the silver bullet, instead it will provide ideas that you can take away for your shop. We do not formally subscribe to any one methodology, but we endeavor to use Agile processes, and we are tweaking the process as we learn from each subsequent release that we work on.

While we have had some great results, the focus of this paper is on the little things that gets us those results. The ideas in the paper are presented in more of a cookbook style where the results are more about the preparation and ingredients, rather than sticking to any pre-determined step-by-step process. The execution of the process is an important step, but without good ingredients, sometimes you get less than stellar results. We are still tweaking the final concoction, but our base recipe is fully cooked. Think of this presentation as family favorite recipe for which you can add your own flair.

Alan Ark is the QA Manager at Compli, in Portland, Oregon. Alan has gained tremendous experience working for Unircu, Switchboard.com, and Thomson Financial - First Call. While at Thomson, Mr. Ark presented “Euro: An Automated Solution to Currency Conversion” at Quality Week ‘99, detailing how an automated test suite written in Perl was used to save the company great embarrassment. Currently, Alan is teaching his staff Ruby to speed up the test process.

Dick Hamlet, Portland State University

Examples of systems built from software components are presented to illustrate the pitfalls of testing component-based designs. A suite of prototype CAD tools for component-based analysis and synthesis is used to measure and predict graphs of component- and system behaviors. Two questions are investigated: (1) To what extent can the results of component testing predict the results of system testing? (2) What component- and system designs minimize surprises that emerge only when systems are assembled and tested?

Dick Hamlet is Professor (emeritus) in the Department of Computer Science at Portland State University. He has worked as an operating-systems programmer and systems-programming manager for a commercial service bureau and for a university data-processing center. He was a member of the software engineering research group at the University of Maryland for 12 years, a visiting lecturer at University of Melbourne in 1982, and a Science Foundation Ireland fellow at National University of Ireland, Galway in 2003-4. He has been actively involved in theoretical program-testing research and in building testing tools for 30 years. He is the author of three textbooks and more than 50-refereed conference and journal publications. Currently he is investigating the theoretical foundations of testing.

Dick has been involved with PNSQC since 1985, when he gave the keynote speech at the 3rd Conference. He has worked on program committees for many of the Conferences since then and he helped to invent the present system for soliciting and selecting technical papers. He also gave a keynote speech at the 14th Conference in 1996 and served on a panel at the 25th Conference in 2007.

Patrick Kua, Thoughtworks

Automated acceptance tests play an essential role in maintaining a certain level of quality into software, capturing regressions in system behavior, and catching integration bugs not caught by more granular unit tests as popularized by tools like JUnit.

The quality assurance and sometimes, though less often, the development parts of a team traditionally own these set of tests, yet there is value is spreading ownership of these tests to other parts of team to help improve quality in many aspects of the software development process.

Automated tests are not new. Using them as a tool to bolster collaboration in teams is less understood and definitely much less undervalued. This paper will look at how building automated acceptance tests the entire team owns can affect the quality of software in very positive ways.

Patrick Kua is an agile coach, facilitator and developer for ThoughtWorks. He has been working with individuals on teams in agile environments for the last four years, and understands how powerful and responsive people can be when working together in a collaborative way. He is always interested in aspects of continuous improvement, and how light weight processes can boost team effectiveness whilst still maintaining high levels of quality.

Jon Bach, Quardev, Inc
Grigori Melnik, Microsoft
Michael Puleio, Microsoft

This talk is the tale of four software professionals who set out to produce guidance for testers around the world. This team, headquartered at Microsoft’s Redmond campus in the Patterns & Practices division, focused on the topic of “acceptance testing.” They set out to collaborate not only with each other, but also with other internal teams. That quickly expanded to testers outside of Microsoft, and borrowing ideas from other domains like aerospace and biomedical to form rich notions of what acceptance testing might mean, no matter the context.

This paper explains the ways we collaborated and strived to ensure that the artifacts we were producing were relevant and meaningful to our target - in effect, “eating our own dog food.” Furthermore, it explains our research about the notion of acceptance testing, which we found comes in two phases: readiness (preparing for the acceptance) and the customer actually performing the acceptance tests.

Jon Bach is Corporate Intellect Manager and senior consultant for Quardev Laboratories (www.quardev.com) - a Seattle test lab specializing in rapid, exploratory testing. He has been a presenter at PNSQC for the last 5 years.

Grigori Melnik, Michael Puleio, and Rohit Sharma are employed by Microsoft Patterns & Practices group.

Doug Whitney, McAfee
Srinidhi Krishnan, McAfee

Working with remote teams, whether across town, or across the ocean, can have its set of challenges. Trying to determine how to interact, when to interact, and if to interact can make the difference between a successful result and a failure. The teams in both the US and India for the McAfee product Total Protection Service have created a successful model that has resulted in on-time delivery and high team morale. There seems to be a common thread that binds the team together - CHOCOLATE. This paper will describe the initial interactions, planning methods, travel tips, team building experiences, and types of chocolate used to enable our successful team experience.

Doug Whitney manages two QA teams at McAfee. He has presented papers at PNSQC and Quality Week on various topics. He has 16 years of QA experience and has managed QA teams at both McAfee and Intel.

Srinidhi Krishnan manages two QA teams in Bangalore India for McAfee. He is involved in several QA initiatives within McAfee’s QA organization. He also managed the corporate applications group for McAfee in Bangalore, India.

Diana Dukart
Brian Lininger, Oracle

Well-executed collaboration can often make the difference between a quality software system and one that falls short. Strong collaboration skills are necessary for the varied roles software engineers are required to apply when undertaking software and information technology projects. In fact, close coordination and communication is needed in all aspects of the software process, from the initial customer requirements definition through the detective-like collaborative work required to triage and resolve errors in the final system. Any flaw in these lines of communication can greatly increase the risk of diminished quality in the end product.

During the process of completing the Oregon Master of Software Engineering (OMSE) Practicum Project, the authors applied a variety of collaboration styles and technologies commonly practiced on software engineering projects today. Project aspects addressed by such practices include distributed team member location, variability of member experience and skills, multiple modes of customer integration, and constrained schedules and resources. This paper examines the “lessons learned” from the full range of collaborative styles and technologies that were encountered during the project.

Diana Dukart is a senior software engineer with professional experience at Tektronix, Inc. She is a recent graduate of the Oregon Master of Software Engineering (OMSE) program at Portland State University and holds a bachelor’s degree in Computer Science from the University of Portland.

Brian Lininger is a senior software engineer at Oracle, where he performs Quality Assurance activities on the Fusion Middleware Application Server. He is a recent graduate of the Oregon Master of Software Engineering (OMSE) program at Portland State University and holds a bachelor’s degree in Computer Science from the California State Polytechnic University at Pomona.

Douglas Hoffman, Software Quality Methods, LLC.

Most automated tests perform the same exercise each time the test is run. They are typically collected and used as regression tests. Testers often think of test automation GUI based scripted regression testing. Tool vendors sell the automating of manual tests. This is a very limited view of the potentially vast possibilities for automating tests. When we think of test automation, we should first think about extending our reach - doing things that we can’t do manually. This topic is about getting past the limitations of automated regression suites and generating much more valuable kinds of test automation.

The difficult part of automation is determining whether the software under test (SUT) responds correctly. Automated tests can easily feed huge numbers of inputs to the SUT. Variation of inputs in automated tests can use data driven approaches or random number generators. In the absence of an excellent mechanism for recognizing expected SUT behavior (an oracle), verification is time consuming and extremely difficult. With an oracle, automated tests can be designed using potentially huge numbers of variable inputs to evaluate the responses of the SUT - without doing exactly the same test exercise each time.

Points the audience will take away:

• Why regression tests are a poor way to find new defects

• What regression tests are good for

• How automated tests using oracles can look for new defects each time

• Eight specific approaches for non-regression automated tests

• The types of defects discovered using these approaches

Douglas Hoffman has over twenty-five years experience in software quality assurance. He has degrees in Computer Science, Electrical Engineering, and an MBA. He has been a participant at dozens of software quality conferences and has been Program Chairman for several international conferences on software quality. He designs test automation environments and automated tests for systems and software companies.

He is an independent consultant with Software Quality Methods, LLC, where he consults with companies in strategic and tactical planning for software quality, and teaches courses in software quality assurance and testing. He is a Fellow of the ASQ (American Society for Quality), founding member of SSQA (Silicon Valley Software Quality Association) and AST (Association for Software Testing), and is a long time member of the ACM and IEEE. He is Past Chair of the Santa Clara Valley Software Quality Association (SSQA) and Past Chair of the Santa Clara Valley Section of the ASQ. He has also been an active participant in the Los Altos Workshops on Software Testing (LAWST) and dozens of its offshoots. He was among the first to earn a Certificate from ASQ in Software Quality Engineering, and has an ASQ Certification in Quality Management.

Claudia Dencker, Stanford University

In February 2007, the Quality Assurance organization in Administrative Systems (AS), Stanford University was formally underway with two mandates in mind: to set up an independent testing function and to improve the quality of AS delivered software solutions. The challenges that these two mandates attempted to address are as follows:

• Inconsistent (and in some cases, non-existent) processes, tools and procedures across all practice areas within AS

• Lack of transparency of project activities within and outside of AS

• Bad software quality (DOA)

Today significant progress has been made on all three challenges. Silo’d practices are slowly merging into a cohesive whole, projects are more open and transparent with business partners actively participating in project progress, and software quality is improving in key areas.

This paper reports on the progress that the QA organization has achieved since its formation and highlights the unique role that collaboration tools and processes have played in improving business partner satisfaction in AS’ delivered solutions at Stanford University. One project will be profiled, the PeopleSoft Student Administration and Human Resources version 9 upgrade project that affects all business offices in support of the university’s mission of teaching, learning and research. Tools used are a wiki integrated with an issue tracker, test case management, project dashboards and more.

The more open mode of working with staff across different job disciplines, in different physical locations and in a few cases, across multiple time zones has led to a much higher degree of satisfaction for all team members. Additionally, the university has benefited where individuals are accountable and the project is more transparent with open collaboration tools and processes.

Claudia Dencker is Director of QA, Administrative Systems, Stanford University. She was formerly Program Coordinator of the newly consolidated program, SEQ (Software Engineering and Quality) at University of California, Santa Cruz Extension and President of Software SETT Corporation (established in 1987), a company that provided QA and software testing solutions to software IT professionals worldwide. She has trained professionals worldwide in software testing and test management as well as led and managed teams on many projects in Silicon Valley. Ms. Dencker graduated from San Jose State University with honors.

Jean Hartmann, Microsoft

Like many divisions within Microsoft, Developer Division, with its two flagship products - Visual Studio and Visual Studio Team System - allocates a significant portion of its time and resources to ensure the highest possible quality of its maintenance releases. Product teams within the division face the challenge of having to execute increasing numbers of automated tests against a growing code base in ever-shorter development test cycles.

In a previous PNSQC conference paper, we described an approach that focused on leveraging so-called selective revalidation techniques using code coverage data, which provided a significant reduction in the number of regression tests that needed to be rerun for a given code change and better prioritized test suites. Thus, the focus of this paper is to describe and discuss how we deployed and evaluated these techniques as part of a recent maintenance test cycle.

Jean Hartmann is currently Test Architect in Microsoft’s Developer Division, having previously held a similar position with the Internet Explorer (IE) team. His main responsibility is driving quality and test innovation topics for the division. His technical interests, while diverse, have gravitated over the years towards three main areas of interest, namely testing, requirements engineering and architectural analysis. Before joining Microsoft, he was Manager for Software Quality at Siemens Corporate Research for twelve years and earned my Ph.D. in Computer Science in 1992.

Kacey Arnold, Adobe Systems
John Van Straalen

What is an open-ended system? At its most basic, a system that no matter how much time you spend testing it you can not test all possible functions and/or variables in all combinations. Much of the software developed today is moving towards being open ended, with the trend towards a rich user experience and user configurable settings.

How do you test, accurately and quickly, open-ended systems? The common answer would be to attempt to test comprehensively all functions in all possible combinations. What happens if you cannot comprehensively cover the system in the time that the release schedule allows for testing? One solution is taking a random sampling of sufficient quantity across the test space, to expose the bug classes that are present in the test space. This approach can be done in a fraction of the time it would take to cover the test space comprehensively. Random sampling is commonly used in real world testing of most large data sets. Examples include random testing of; people at a security check point, circuit board production, food quality, or any other large data set.

Our paper will discuss the methods and uses of random sampling to uncover classes of bugs within the test space in a reasonable time frame. We will be using real world examples from our experiences with large open-ended systems to show how this method can be used in released software. This paper will have some light use of statistics and elementary set theory to support our position and examples. We will also cover some methods that have proven useful in getting managements support of this testing method.

Kacey Arnold has 8 years testing, lead, experience building and designing automation test frame works and harness’s on enterprise and SaaS software and holds a Bachelor Of Science in Applied Math.

John Van Straalen has 6 years testing experience in automation for enterprise systems and holds a Bachelor of Science in Computer Science.

Lanette Creamer, Adobe Systems, Inc.

Customers who are so delighted with a product or service that they will tell others are a magnificent asset which every company wishes to retain and attract. While we have many ways to test for code stability, features, and functionality, there are not as many test methodologies that explore reporting on the overall user experience.

Workflow testing is the practical approach that we have applied to testing collaboratively for the user experience. Workflow testing methodology scales from a use case all the way to testing across the Adobe Master Collection for CS3. Having an efficient way to report holistically the user experience is an effective way to improve overall product quality. This is especially true in multi-user scenarios which span many features, different applications, and a myriad of operating systems.

Lanette Creamer has been with Adobe Systems since 2000 testing products such as Adobe InDesign versions 1.5 through CS, Adobe InCopy 1.0, Shared Technologies across applications like XMP, XML, WebDav, and has been working as the Quality Lead for the Adobe Creative Suites Workflow Team since 2006. Lanette studied Graphic Design at Western Washington University, but her true love was for Photoshop, Illustrator, and PageMaker. After attending an inspiring seminar at the CAST conference in 2007, she started a testing blog at http://blog.testyredhead.com/ hoping to find other people who are passionate about collaborative testing.

Celeste Yeakley, Education Finance Partners
Jeff Fiebrich, Freescale Semiconductors, Inc.

This paper provides a human approach for building effective collaborative process improvement techniques. Rather than relying on specialized quality departments, these methods engage each employee in the process of delivering a quality product by collaboratively updating tasks and activities to include essential improvement elements. This laymen’s approach has been effective in building improvement processes into the daily work life of every worker. The paper includes:

• Champions for collaboration

• Training effort collaboration

• Cultural diversity collaboration

• Managing change

• Encouraging continuous process improvement

• Spreading the improvement initiative via collaboration and “sneezers”

Rewarding and recognizing process improvement successes - every company needs a strategic reward system for employees that address these four areas: compensation, benefits, recognition, and appreciation. The problem with reward systems in many businesses today is twofold: They are missing one or more of these elements (usually recognition and/or appreciation), and the elements that are addressed are not properly aligned with the company’s other corporate strategies.

Getting behind your customer’s eyes to see their view of quality - quality is in the eye of the customer, and as your customer changes, your measure of Quality will likely change. You need to be careful not to fall back into the rigid, unchanging processes. As soon as you are certain you have decided on the most effective process and the method to document it, someone will find a way to improve it.

The paper’s approach and style evolved from the authors’ hands-on experience in software, semiconductor, and computer industries. The paper contains summaries, aids such as checklists, templates, exercises, tips and pitfalls to facilitate quick execution of the topics discussed. Most importantly, the paper addresses collaboration in human terms and gives the reader real world examples that are tangible and understandable to anyone regardless of their role in an organization.

Celeste Yeakley is an organizational leader with more than twenty years of experience in software/systems engineering and process improvement. She is a broad-based team lead and participant, with proven skills that focus teams by applying creative and innovative business planning processes. She is accomplished in business strategy, business planning, team/project management (certificate in Software Project Management), software capability enhancements, and organizational leadership. As an internal assessor, she either participated or led software process evaluations for organizations in the United States, Brazil and Russia. A University of Texas at Austin graduate with a Master’s degree in Science & Technology Commercialization, Celeste has collaborated at small start up companies as well as large corporate giants such as Dell and Motorola. She has contributed to her profession by serving on the UT Software Quality Institute board from 1993-2003 as well as in community quality assessments. Her passion is for helping people understand how to work together using established frameworks like CMMI and ISO to their best advantage. She encourages lateral thinking and uses every day examples to drive people to understand the big picture and how it fits into their software worlds.

Jeff Fiebrich is a Software Quality Manager for Freescale Semiconductor Inc. He is a member of the American Society for Quality (ASQ) and has received ASQ certification in Quality Auditing and Software Quality Engineering. He is also an RABQSA International Certified Lead Auditor and has achieved CMMI-Dev Intermediate certification. He has previously worked as a Quality Manager for Tracor Aerospace in the Countermeasures Division. A graduate of Texas State University with a degree in Computer Science and Mathematics, he served on the University of Texas, Software Quality Institute subcommittee in 2003 - 2004. He has addressed national and international audiences on topics from software development to process modeling. Jeff has over twenty years of Quality experience as an engineer, project manager, software process improvement leader, and consultant. As both an internal and external consultant, he has played significant roles in helping organizations achieve ISO certification and Software Engineering Institute (SEI) Maturity Levels. He has led numerous process improvement initiatives in many areas of software engineering including project management, employee empowerment, software product engineering, quantitative management, training program management, and group problem solving. Jeff has worked extensively on efforts in the United States, Israel, Europe, India, and Asia.

Celeste and Jeff are the authors of the recently released book, ‘Collaborative Process Improvement’, Wiley-IEEE Press, 2007.

Debra Lavell, Intel

At Intel, we have implemented a more effective approach for capturing and sharing lessons learned through retrospectives. We will explore the people side of bringing retrospectives into an organization. The tips and tricks are from a facilitator’s perspective. We will share how one deals with the human aspects of introducing collaborative change.

As with any organization or business process change undertaking, one of the most difficult challenges to overcome is getting an entire company to change their culture and modify the way they work and behave. Introducing the retrospectives methodology into Intel has been no different. A key learning we’ve discovered is the services of a trained facilitator is critical to improving the likelihood of sustainable change. In this paper, we explore the people side of bringing retrospectives into an organization. The tips and tricks are from a facilitator’s perspective. This paper will explore how one handles the human aspects of introducing collaborative change.

Debra has over 10 years experience in quality engineering. She currently works as a Program Manager in the Corporate Platform Office at Intel Corporation, focusing on Retrospectives and Organizational Learning. Since January 2003, Debra has delivered over 200 Project and Milestone Retrospectives for Intel worldwide. Prior to her work in quality, Debra spent 8 years managing an IT department responsible for a 500+ node network for ADC Telecommunications. Debra is a member of the Rose City Software Process Improvement Network Steering Committee. For many years, she was responsible for securing the monthly speakers. She currently is the President of the Pacific Northwest Software Quality Conference, Portland, Oregon. She holds a Bachelor’s of Arts degree in Management with an emphasis on Industrial Relations.

Ian Knox, Skytap

Virtualization is a groundbreaking technology that promises quantifiable benefits for application development and QA organizations: faster lab deployment, less manual set-up work, greater resource flexibility and utilization, and easier reproduction of defects.

However, adopting virtualization in a development or QA organization is not without issues. Often it is not obvious whether to build out a custom virtualization framework or make a strategic bet to implement a full virtual lab management solution, complete with automation and a pool of centralized hardware.

This paper discusses the software quality challenges commonly faced by application development teams and how virtual lab automation can lead to a more strategic approach to QA practices. It describes the best practices for virtual lab automation adoption and also highlights the common pitfalls organizations face during implementation. Finally, this paper outlines the steps to evaluate a virtualization solution for your QA organization and provides further resources to help you get started.

Ian Knox is the Director of Product Management where he has been responsible for all aspects of Skytap’s product management and go-to-market strategy. Ian has spent over 15 years in the software development industry, including numerous positions in software development and product management. Ian joined Skytap from Microsoft Corporation where he was group product manager for Microsoft Visual Studio. In this role, Ian led the product management team responsible for introducing Visual Studio Team System, an industry-leading application development and testing platform. Prior to Microsoft, Ian was a Principal Consultant at PricewaterhouseCoopers where for 7 years he worked on global software delivery projects for Fortune 500 clients.

Vaibhav Mittal, Adobe Systems
Abhishek Talwar, Adobe

Development of a product is a complex task. It involves multiple modules, multiple language support, multiple configurations, multiple footprints etc. As a result, the testing of a product also becomes very complex. The need of the hour is to “smartify” your testing. Over the last few years, the essence of testing has shifted from ‘execution’ to ‘tracked execution’. If one can track the execution, one can take real time decisions to improve the quality of the product. In addition, one can help the system work for us and thereby increase our resource count.This paper introduces a tool we named TestMaster System (TMS). TMS keeps a track of all the test cases, can fire automation runs, can report results, has customized view depending on user privileges, and is integrated to the bug base. The smart aspect of TMS is that it can adapt its interface to any environment. Salient Features of TMS:

• Features a web based repository to include all the desired aspects of a test case.

• TMS integrates with automation frameworks. A single click can trigger complete automation.

• TMS employs a schedule-creation criterion established by the management team to assign tasks related to bug assignment (developer/tester) and tracking test case results.

This paper will provide avenues on how to drive a software development life cycle through use of TMS. Features of existing products in the test management domain are taken as baseline and new features that evolved as a result of our experience are added to evolve a future TMS.

Abhishek Talwar is Lead Software Engineer at Adobe since last 4 ½ years. He has over 5 years in Video, Imaging and Telecommunications. He has worked on various fields of testing including White Box testing, Automation, scripting and Black Box testing. His current role involves developing tools thta make the life of a tester, at Adobe, easy. Abhishek holds an international patent to his name in the field of video. He also holds a patent publication in the field of Documents. He has written five international paper publications in the field of API testing, Scripting, testing best practices, and project management. One of his submissions API Testing: Catching hidden bugs early in cycle was presented at the STC2005 main conference in Hyderabad. One of his papers on Quality mantras was voted as the best paper in Adobe India QE summit, 2005 and chosen as an article in QE quarterly newsletter.

Vaibhav Mittal is Software Engineer at Adobe Systems. He has 3 years in the field of Software development. His role in Adobe is that of a solutions developer to provide internal tools for different product teams. He has developed tools on various platforms that support more than 20 products inside Adobe. He has also been part of a software development firm that develops software for a Japan based graphic imaging firm. Vaibhav holds a B. Tech degree (Hons.) in Computer Engineering from The Technical Institute of Textile and Sciences, Bhiwani.

Brian Rogers, Microsoft

“How many testers does a software project need?”

Ask a test manager and you will likely hear “As many as we can find!” Unfortunately, it is often difficult to staff a testing organization with as many resources as is optimal. Does this mean a project must suffer poor quality if it falls short of the magic “1:1″ test-to-dev ratio? Absolutely not!

Projects with smaller test teams are forced into certain behaviors out of necessity. As it turns out, these behaviors can be great assets in the quest for high quality. Some of these behaviors include customer-focused planning, aggressive trimming of unneeded functionality, and balancing of traditionally test-assigned tasks across the team. While these are almost requirements in smaller organizations, teams of any size can learn and benefit from these activities as well.

A key factor for success given a small test team is the recognition that quality is everyone’s responsibility. Testers and developers must be partners, not rivals, in ensuring quality.

Brian Rogers has worked as a software development engineer in test at Microsoft for five years. For most of his career, he has focused on distributed testing of enterprise messaging systems, including a three-year assignment on the first version of Windows Communication Foundation. He is a strong proponent of engineering excellence within the test discipline, and has designed and developed static analysis tools for detecting defects in test artifacts. Brian has a Bachelor of Science in Computer Engineering from the University of Washington.

Kacey Arnold, Adobe Systems
John Van Straalen

Unless you live under a rock you have heard of test automation - we have read papers on its uses, seen vendor ads, and heard stories from the industry about it. We have heard stories about how it saved the day and how it was an utter failure. Some see test automation as a silver bullet; others claim it is a waste of time. The reality is that test automation is a tool, when used appropriately, is a worthwhile effort. However, how do you determine if it is worth the cost to setup, build, and maintain? In short - how do we make it pay for itself?This paper will cover the general strengths and weaknesses of automated testing, as learned by the authors from their direct experiences, the costs to be considered (both upfront and long-term) when considering automated tests, and how to use automated testing in such a way that it pays for itself.

Kacey Arnold is a Quality Engineer at Adobe Systems. She has a Bachelor of Science in Applied Math.

John Van Straalen is a QA Lead at Audio Precision. He has a Bachelor of Science in Computer Science.

Charley Baker, Gap Inc. Direct

Gap Inc. Direct has gone through a major paradigm switch in the past several years. The transition to Agile, affected the way project teams and disciplines work together. Changing methodologies has involved a change in mindset, allowing our teams to work together more effectively while creating new opportunities in toolsets and testing practices.Now the lines between developer and testers are blurry. Developers, Technical Managers, QA, including offshore teams, all go through an induction into agile and additional quick training in our QA processes, basic ruby programming, and testing guidelines.Switching automation tools enabled the company to move to meet the demands of an increasing number of projects with all QA Engineers able to write automated tests to cover our regression suites, freeing up time to do more exploratory testing. Our regression tests are now run through Cruise Control under Continuous Integration, allowing us to quickly get feedback on breaking tests, whether due to application or test code failures.

QA Engineers now work closely with developers, reducing previous automation problems by constant communication, writing tests on the same timeline as the development code is written and in cases tests have been written and finished before development code is ready. The tight relationship between developers and testers has also enabled us to work with them on changes that make our applications more testable - additional APIs to feed test data requirements, simple DOM changes to more easily access controls. Now that developers have exposure to functional testing, they have the understanding and empathy to work with QA in a more cooperative manner.

Charley Baker has over 15 years of experience in the Software Industry in both Development and QA roles. He has worked at Gap Inc. Direct for over 6 years where he’s currently in the position of QA Architect. He has worked both in development and testing, and helped transition Gap Inc. Direct into the Agile space. Charley is the Project Manager and a core committer on Watir, and a co-presenter at Agile 2007 with Bret Pettichord.

Jagannathan Venkatesan, Microsoft
Manuel Merino, Microsoft
Craig Merchant, Microsoft

There have been numerous efforts in an attempt to solve the problem of cross-browser support in the context of web UI testing. Some examples are: using JavaScript driven automation and solutions like WebRunner. The lack of low-level access to DOM elements required for operations such as retrieving all their properties and not being able to trigger Windows/Operating System Events directly over the browser (for example, simulating mouse move events on the browser) is problematic. Further, the tests browser and methods need to be agnostic, allowing the same tests to run in IE (via mshtml or Microsoft UI base automation) as well as Firefox and other browsers via AjaxBrowserLayer with no code changes necessary.

This paper describes a test harness for developing test automation for websites. The harness provides a set of classes that a test developer can use to develop custom coding solutions for controlling (automating) and verifying websites. The harness is completely browser agnostic meaning a single code base can execute on multiple browsers.

Bios not yet available.

Diana Larsen, FutureWorks Consulting LLC
Sharon Buckmaster, FutureWorks Consulting LLC

Companies interested in gaining software quality through collaboration maximize the talents of their female software developers, testers, business analysts and quality assurance staff. Although women’s participation is on the rise in many fields, including some of the traditionally male-dominated ones such as accounting and medicine, the percentage and number of women in the IT field is actually declining. The Computing Research Association reports fewer computing degrees awarded to women in 2004 than in 2000. Numerous academic and industry studies have documented that high exit rates for women from the IT arena contributes to an inability to fill roughly 500,000 information technology jobs nationally. With more than 50% of the current U.S. science, technology, and engineering workforce approaching retirement age, organizations must examine strategies to address the workplace conditions that attract capable women and men, and increase the likelihood of their continued employment.

Catalyst, a leading research and advisory organization, works globally with businesses to expand opportunities for women and business. In their 2007 landmark study on Women in IT, Catalyst examined drivers of satisfaction, retention, and advancement among women in technology. Learn to leverage these six drivers to recruit and retain talented women for your software development projects through an interactive discussion exploring which drivers make the most sense for your organization.

Sharon Buckmaster, Ph.D. and Diana Larsen are the principals of Futureworks Consulting, a firm specializing in bringing collaborative processes to organizations that want more productive, resilient workplaces. Both Sharon and Diana have many years of experience developing the generative capacities of individuals and teams that lead to higher quality products and services as well as a higher quality of organizational life.

Diana is known in the software industry for conducting project retrospectives and transitioning groups to Agile processes. She currently chairs the board of the Agile Alliance. Her publications include Agile Retrospectives, Making Good Teams Great, coauthored with Esther Derby. She consults and speaks internationally.

Sharon Buckmaster is a Ph.D. with expertise in executive leadership and gender-equity issues in organizations. Her research has focused primarily on women in leadership roles. She is the founder and past president of The Women’s Center for Applied Leadership and is affiliated with the Center for Gender in Organizations at Simmons College. Sharon teaches in the Masters-level Applied Information Management Program at the University of Oregon and coaches executives and upper level managers.

Joy Shafer, Microsoft

About eight months into our first Agile development projects, the Development Lead, the Lead Program Manager, and I, the Test Lead, attended a seminar titled “Agile meets Offshore.” The presenter, a gentleman with considerable experience in this arena, told the audience not to try Agile offshore on “… first releases of complex and high-technology-risk projects, if your onshore development process is not in place, [or] if you don’t have any onshore Agile experience.” My colleagues and I looked at each other, and the Dev Lead said, “No wonder this is so hard. We’re doing all of those things!”Six months later, we had fallen into a cadence of releasing both of our online services on time every two months. These were the easiest releases I’ve experienced in my fifteen-year career in software testing. We were even able to release a week early in one case. Our software quality and team morale was high, we worked productively in round-the-clock shifts, and the offshore teams were truly an extension of the core team.

This paper details our journey from chaos to concord. It highlights the challenges and successes we experienced while developing software as a service with a globally distributed team using Agile methodologies. We concurrently developed two small, first-version online services using a development team that was located in Redmond and Moscow, and a test team that was located in Redmond and India. We encountered many interesting and difficult challenges, but were able to successfully overcome them and release high quality software on time, delighting our stakeholders. This experience report highlights our learnings, focusing on several critical success factors, such as well-defined processes, cultural awareness, continuous integration, open communication, and relationship building.

Joy Shafer is currently a Senior Test Lead at Microsoft on the Mobile, Voice and Partner Services team. She has been at Microsoft for three years, lending her expertise to developing and testing online services. Prior to Microsoft, she tested and managed testers at Quardev, NetManage, STLabs, and Aldus Corp. She also was her own boss for a time, teaching and consulting in the area of software testing methodology. Joy is an active participant in community QA groups. She holds an MBA in International Business from Stern Graduate School of Business (NYU).

Diana Larsen, FutureWorks Consulting LLC

Trust forms the bedrock of effective software teams. Trust allows teams to communicate quickly and to respond rapidly to changes in the project. Without sufficient trust, team members waste effort and energy hoarding information, forming cliques, dodging blame, and covering their tracks. A climate of trust provides a foundation for effective team processes, adaptability, and high performance. How can we shatter the deep-seated cycle of distrust in many organizations and help this essential trust emerge? Team leaders can stimulate and accelerate trustworthiness and trusting among team members, and between the team and its stakeholders by paying attention to membership, interactions, credibility, respect, and behaviors. In this session we’ll investigate ways to accelerate trust-building within teams, including a definition of professional trust, a model for team interactions that leverages trust, ways to recognize when a team has “trust issues,” and skills that help teams develop greater trust.

See Diana’s bio for P-36.

Diana Larsen, FutureWorks Consulting LLC

Software teams experience what goes right and what goes wrong, what works and what doesn’t, during each and every day of each and every project. What happens to that hard-earned experience? In too many organizations, on too many projects, it dissipates as team members shift focus and move on. Real value lies in capturing, managing, and disseminating technical knowledge and process wisdom to improve current and future projects.

Retrospectives offer the greatest lever for improving any software development project or process based on the solid data of a team’s immediate past experience of success and failure. Through Retrospectives, teams systematically evaluate their own performance, explore their lessons learned, expand their capacity and capability, and choose how to improve the way they build and deliver products to your customers. For best results, smart teams and organizations convene Retrospectives iteratively throughout the project and again after delivery. Successful teams learn how to encounter problems and implement solutions effectively throughout the project, not just at the end, and identify ways to maintain the relevance of continuous improvement to the work of the team.

See Diana’s bio for P-36.

Kal Toth, Portland State University

When embarking on a new project, the software engineering manager will need to decide early on whether to follow a Waterfall, Agile, Prototyping, Incremental or some hybrid or variant of these software processes. To assess project feasibility, to secure budget, and to properly plan resources and schedules, responsible managers should also decide about their software estimating process - whether to us expert judgment with estimating rules-of-thumb; parametric techniques like COCOMO and Function-Points; or estimating databases populated with analogies and proxies from prior projects. The characterizing attributes of a given new project greatly influence software process and estimating choices.This paper provides context and guidance that will help the software practitioner understand the influences of project attributes on the selection of suitable software processes and on software estimating techniques when embarking on the next software project. This paper will also assist companies decide how to best apply their resources to maintain a suitable software estimating infrastructure in support of project planning and execution.

Kal Toth is the Director of the Oregon Master of Software Engineering Program (OMSE) and Associate Professor in the Maseeh College of Engineering and Computer Science, Portland State University (PSU). A Professional Engineer (P. Eng) with a software engineering designation registered in British Columbia, he has a Ph.D. from Carleton University in electrical and computer systems engineering. Kal has over 25 years of management, technical, and consulting experience leading and working for a range of technology companies and organizations including Hughes Aircraft, Datalink Systems Corp., BC Software Productivity Centre, the CGI Group Inc., Intellitech Canada Ltd., National Defence (Canada), Communications Canada, and External Affairs (Canada). He has managed and participated in a technical capacity addressing project management, software quality, and information security aspects of air traffic control, e-commerce, distributed information, and packet-switching systems. At PSU, he delivers software engineering courses covering software project management, software quality, and practicum projects. He is conducting research in the field of identity management and security.

Ian Savage, McAfee, Inc

Between three different sessions at the Agile Open Northwest 2008, the author and another software professional developed a cost-benefit charting scheme that:

• Enables development teams and customers to prioritize features,

• Identifies features that should be quashed, pursued, or further decomposed, and

• Allows stakeholders to monitor and manage priorities over the course of development.

This collaboration amounted to about 15-20 minutes total and produced a diagramming technique that can be used in bleeding-edge agile shops and in shops using more traditional methods. The chart is a basic Cartesian plane with Cost (Points) on the x-axis, Value on the y-axis, and something other than dots at the intersections. We call this technique Analyze This as it focuses analysis where it is needed.

My collaborator is a noted agilest with direct control of his team’s practices: I am a software quality practitioner with influence in a more traditional plan-driven shop.

A quality evangelist, Ian is a veteran software developer, quality assurance engineer, and manager with experience in the manufacturing, financial services, construction estimating, and security domains. For more than 30 years, he has worked to improve productivity and software quality through rigorous development methods and processes and now through the pragmatic application of Agile methods.

Ian serves on the Software Association of Oregon’s Program Committee. He authored the SAO Quality Assurance Special Interest Group charter and serves on the SAO QASIQ Steering Committee. He has contributed to American Society for Quality’s certification program for software quality engineers and the Software Engineering Institute’s Software Engineering Process Group conference. He is a member, and supporter, of the Agile Alliance.

Ian attended the very first Pacific Northwest Software Quality Conference in 1983. Since then he has served on PNSQC’s board as President, Vice President, and Secretary. He has also chaired the PNSQC Software Excellence, the Strategic Planning, and the Program Committees. Ian is currently serving as the PNSQC Program and Conference Chairman.

Ying Kwong, State of Oregon

This presentation discusses the management of outsourced information technology (IT) projects. Project management (especially quality/risk management and development lifecycle considerations) will be discussed from the perspectives of enterprises acquiring software systems. Examples from the authors’ experience in providing IT investment oversight with the State of Oregon will be discussed, with emphasis on management issues and the role of independent QA. This presentation deals with quality management challenges during design, development, and implementation of software systems in large enterprises, emphasizing management (not engineering) issues.

Ying Kwong is the IT Investment Oversight Coordinator with the Oregon Department of Administrative Services. He is a member of the staff of the State CIO. He has previously held management and R&D positions in high tech companies. He is a Project Management Professional and received the PhD in applied physics from Cornell University. He is an adjunct faculty member of the School of Business Administration at Portland State University.

Karen Johnson, Software Test Management Inc.

In a fast-paced world with volumes of data thrown at us each day, it’s hard to find meaning and relevancy in the data presented to us. Our project stakeholders face the same challenge of trying to sift through data to find meaning. Storytelling techniques help us take facts small and large from our testing experiences and weave together information in a format that brings data to life - the story.

This paper describes how to use storytelling techniques to report experiences with the product under test and learn how to twine facts both small and large to together to provide more meaningful product reports. Examples of presenting product status in story form will be shared.

Sharing information in the story form helps with comprehension. We might think of storytelling from our childhood days and envision our time-starved, impatient stakeholders as being unreceptive to a story when what they demand is hard facts. But there are storytelling techniques that we can use to present information in such a way that we can resolve questions and illuminate meaning without building fables.

Karen N. Johnson is a software testing consultant in Chicago, Illinois, USA. Karen views software testing as an intellectual challenge and believes in the context-driven school of testing. She has extensive experience in software testing and test management. Karen frequently speaks at software testing conferences. She has presented at STPCon, CAST, PNSQ, StarEast, and StarWest. She’s also presented at several local quality group meetings such as IQAA, CQAA, and NOSQAA. She publishes articles on software testing and has been published in Better Software, InformIT and StickyMinds.com. Karen is an executive board member for the Association for Software Testing (AST). She is program co-chair for CAST 2008, the Conference for the Association for Software Testing. Karen is a hosted software testing expert on Tech Target’s website, searchsoftwarequality.com.

Karen Johnson, Software Test Management Inc.

If you need to build a test strategy for a project, this paper will offer ideas on how to get started by providing an in-depth look at what elements a strategy might include as well as how to solicit ideas from other leads and members of your project team. Practical recent experiences on building test strategies will be shared and explored.Learn how to brainstorm, build, and implement a test strategy including elements such as:

• Project Scope

• Product

• Context

• Project Stakeholders

• Risk Analysis

• Types of testing

• Test Environment

• Test Data

• Resources

• Estimates

• Project Plans & Milestones

Learn how to update your strategy throughout the project and to discuss and gain input and acceptance throughout your organization.

See bio for P-48.

Lian Yang, SSD/Microsoft

Test automation itself is a software engineering project and its effectiveness, efficiency, and maintainability quite often present the same engineering challenges as other software projects. The development community has been embracing the “design pattern” concepts for 15 years, in effort of addressing software engineering difficulties and formalizing design and coding process. Design pattern practice is not widely used in the software test automation community. Ill-planned, and roughly designed test automation projects are still widespread. This paper tries to define a “test case automatic generation” design pattern, which addresses the most common and fundamental test automation engineering issues facing most testers today. The concepts presented in this paper have been implemented by the author and his team when testing several Microsoft products.

This paper illustrates how to apply design pattern to test automation design. The pattern is test case generation using micro-action, meta-data and actual data generator, and test state manager. Using this design pattern provides set a good platform for creating random test cases according to the product specification, improving test coverage, and test automation effectiveness.

Lian Yang joined Image Builder Software as a software engineer after graduating from Portland State University in 1991. He worked on software for children including the popular such as KidPix and TreeHouse. He joined Microsoft 1n 1995 and worked at Microsoft Internal Tools for testing and performance tools. There he became interested in software quality issues such as test automation and testability. He became an SDET and joined Smartphone team, where he invented two test automation tools that are critical to wireless application testing. Both inventions were submitted for patents; one has been approved already. Later Lian joined the Windows Security team where he was responsible for distributed test environment building. He joined the Windows Storage Server team as an SDET lead in 2007.

$475

$550

$400

$625

$750

$525

$950

$1100

$800

Playing Nice in the Sandbox

Janet Gregory

In the beginning of the agile discussions, developers and customers reigned over the world. Over the years, testers have raised their heads and said, “We want to be involved. We think we have a place in this agile world!” Well, testers have found a place in many agile teams, but not in the same way as the testers of old. Instead of being the “quality police,” they have become a conduit for providing feedback to the team. Collaboration is the key to a healthy working relationship. Teams that are moving from a traditional phased approach where developers and testers are separated and have a throw it over the wall mentality may struggle with the concept. Janet will share tips for collaboration between developers and testers, how they can learn from each other, and how together, they contribute to make a successful team.

Janet Gregory is a Calgary-based consultant specializing building quality systems and her passion is promoting agile quality processes. She has helped to introduce development agile practices into companies as tester or coach, and has successfully transitioned traditional test teams into the agile world. Her focus is working the business users and testers to understand their role in agile projects.

She is currently writing a book on agile testing with Lisa Crispin due out early 2009. Janet has presented at the Agile and StarWest conference several times and is active in the Agile Testing community.

Selling Your Ideas to Management

Steve Smith

Do you wish you could break through and communicate your ideas more effectively with management? Employees do their best to sell their ideas to managers. Nevertheless, a typical outcome of these conversations is hearing the word “No”" or “Let me think about it.” Employees seldom ask and management seldom offers feedback about the elements of a conversation that worked and those that did not.

Discover three top characteristics of proposals that do work. Hear how others have used these characteristics to transform a conversation with management into a discussion. Management is busy. You may have only one shot at selling your idea. Learn how to effectively target your proposal so that it’s heard and acted on.

Steven M. Smith is a management consultant who helps managers make effective decisions about satisfying customers, managing change, and strengthening teamwork. With more than three decades of experience as a thought leader in technical organizations, he shares his know-how through his writing, consulting, and workshops. He is a founder and host of the annual Amplifying Your Effectiveness (AYE) Conference, at which he leads experiential workshops.

It’s Not Just an Update: Using Status Reporting to Expand Collaboration

Mike Kelly

As a tester, you have undoubtedly encountered the question, “How’s your testing coming along?” If you are anything like I use to be, that question sometimes brings out some interesting mumbles, shrugs, and excuses for meetings “I’m late for.”

A couple years ago, I developed a heuristic to help me deal with questions of status, and now I answer status questions with confidence. In this talk, we will cover the key elements of increasing team collaboration using status reports. We will also look at some collaborative techniques for practicing status reporting, to better develop the skill over time.

Mike Kelly is currently a Software Development Manager for Liberty Mutual, a Fortune 100 company. Mike also writes and speaks about topics in software testing. He is currently the President for the Association for Software Testing and is a co-founder of the Indianapolis Workshops on Software Testing, a series of ongoing meetings on topics in software testing, and a co-host of the Workshop on Open Certification for Software Testers. You can find most of his articles and blog on his website