Objectives
- Create a benchmark and measure the performance of multiple computers.
- Define criteria for measuring the performance of a computer
- List, in order of performance, all computers evaluated by the class.
Attention: This exercise is individual, with a collective final result. You must carry out the measurements and put them online by March 10. The report can only be made after this date.
Before you start
You should remember the concepts of performance measures studied in MC722. Here are some questions to guide your exercise:
- What is a benchmark?
- How many programs should there be in a benchmark?
- Which measure should be used to say that one program / computer is faster than another?
- How to obtain this measure?
- Do you know the difference of the 3 time measurements returned by this command?
- Is the measure used reliable?
- Is the scale used reliable?
- Is there an error in these numbers? How to minimize it?
- What does it mean to say that one computer is X times faster than another?
- Do you know how to differentiate a sequential from a parallel program?
- If your program is sequential, does it make a difference to run it on a parallel machine?
- If your program is parallel, how do you measure your performance on a sequential machine?
You should know how to answer all of these questions, but it is not necessary to put the answers in the report. Do not write the report without knowing the answers to these questions.
Activity
This exercise is the result of collective work, where each component must deliver a report and perform a part of the activity. Each student must perform only one
the following activities:
- Define a program to measure the performance of a computer: Choose a program (it cannot be part of an existing benchmark) that can be used to measure the computer's performance. Activities and recommendations:
- Run the program on a computer in room 302 to take your measurements
- Do not use a program that is, by itself, a benchmark
- The program must have an open source code to allow it to be compiled and run on multiple machines (you do not need to run it on multiple machines). You must provide procedures for installing the programs
- Define, if necessary, a set of standard input (s) and output (s) for your program, so that all executions are identical in behavior
- Prepare a script or define a command line so that the program runs with the correct settings (if any) and parameters
- Be careful not to choose programs that can generate performance numbers that are susceptible to big errors (very small numbers, for example). Runtimes of less than 10s can cause errors in accuracy and therefore will not be accepted as measurements on the computer in room 302 (do more to increase your time).
- Define your performance and measurement criteria on a base computer in room 302. It is necessary to measure performance more than once and consolidate the measurements. Define how many times it has to be measured. If using time, specify which time to use
- Publish the information above so that other students in the class can reproduce their data on other computers. Use this spreadsheet to put your data. Describe the programs in this document.
- Justify the reason for choosing this program, describing it briefly
- Fill in spreadsheet spreadsheets: Fill in blank cells in the spreadsheet, placing the results of running the programs. Place the performance information on the same page described in item 1. The cell for the computer in room 302 is reserved for those who defined the program.
- If necessary, add new computers to the spreadsheet.
- If you include a personal computer, you must give explicit instructions to other students about the times when it will be available (bring it to the classroom, for example, and let your classmates use it). Measures of just 1 student on a computer will be worth less grade.
Based on the results available in the spreadsheet, make your report comparing the performance of all computers. Use all points provided
(You can question the quality of certain measures if you think it is necessary and exclude them). Do not forget to inform, for each measurement, the settings of the computer that was used. Sort computers in order of performance.
Delivery
Send a report of just 1 page describing the activity performed. Use a section / subsection only to describe your individual activity. Analyze and comment on the result.