← Index
NYTProf Performance Profile   « line view »
For split.pl
  Run on Thu Apr 20 02:05:47 2023
Reported on Thu Apr 20 18:31:09 2023

Filename/home/hejohns/documentsNoSync/22f/490/gradescope-utils/bin/split.pl
StatementsExecuted 665651 statements in 6.63s
Subroutines
Calls P F Exclusive
Time		 Inclusive
Time		 Subroutine
133035115.84s53701smain::::__ANON__[split.pl:90] main::__ANON__[split.pl:90]
17295781252.48s2.48sUNIVERSAL::::isa UNIVERSAL::isa (xsub)
133035111.46s53563smain::::__ANON__[split.pl:88] main::__ANON__[split.pl:88]
11199.4ms218msmain::::BEGIN@8 main::BEGIN@8
11111.8ms14.2msmain::::BEGIN@29 main::BEGIN@29
1117.70ms19.1msmain::::BEGIN@24 main::BEGIN@24
1115.42ms6.00msmain::::BEGIN@32 main::BEGIN@32
1114.78ms28.3msmain::::BEGIN@30 main::BEGIN@30
1114.66ms13.8msmain::::BEGIN@18 main::BEGIN@18
1112.16ms2.37msmain::::BEGIN@14 main::BEGIN@14
1111.94ms3.10msmain::::BEGIN@26 main::BEGIN@26
1111.66ms27.1msmain::::BEGIN@33 main::BEGIN@33
1111.64ms7.14msmain::::BEGIN@19 main::BEGIN@19
1111.46ms2.57msmain::::BEGIN@16 main::BEGIN@16
1111.22ms4.73msmain::::BEGIN@21 main::BEGIN@21
1111.03ms14.7msmain::::BEGIN@48 main::BEGIN@48
111896µs1.00msmain::::BEGIN@15 main::BEGIN@15
111860µs894µsmain::::BEGIN@43 main::BEGIN@43
111832µs2.22msmain::::BEGIN@22 main::BEGIN@22
111770µs1.27msmain::::BEGIN@5 main::BEGIN@5
111669µs2.39msmain::::BEGIN@12 main::BEGIN@12
111625µs673µsmain::::BEGIN@6 main::BEGIN@6
111508µs7.14msmain::::BEGIN@20 main::BEGIN@20
111470µs1.28msmain::::BEGIN@50 main::BEGIN@50
111410µs598µsmain::::BEGIN@35 main::BEGIN@35
111404µs60.6msmain::::BEGIN@51 main::BEGIN@51
111164µs2.06msmain::::BEGIN@4 main::BEGIN@4
111106µs151µsmain::::BEGIN@41 main::BEGIN@41
1212651µs51µsUNIVERSAL::::VERSION UNIVERSAL::VERSION (xsub)
401118µs18µsmro::::method_changed_in mro::method_changed_in (xsub)
11117µs17µsmain::::BEGIN@3 main::BEGIN@3
11117µs18µsFcntl::::O_NOINHERIT Fcntl::O_NOINHERIT
413117µs17µsInternals::::SvREADONLY Internals::SvREADONLY (xsub)
11113µs13µsmain::::CORE:readline main::CORE:readline (opcode)
106612µs12µsUNIVERSAL::::can UNIVERSAL::can (xsub)
1118µs34µsmain::::BEGIN@11 main::BEGIN@11
1116µs7µsFcntl::::O_EXLOCK Fcntl::O_EXLOCK
1116µs315µsText::CSV_XS::::BEGIN@1240 Text::CSV_XS::BEGIN@1240
1116µs6µsFcntl::::O_TEMPORARY Fcntl::O_TEMPORARY
1114µs12µsmain::::BEGIN@44 main::BEGIN@44
1114µs16µsmain::::BEGIN@42 main::BEGIN@42
3114µs4µsPerlIO::Layer::::findPerlIO::Layer::find (xsub)
1112µs2µsmain::::CORE:pack main::CORE:pack (opcode)
1112µs2µsmain::::BEGIN@7 main::BEGIN@7
111700ns700nsmain::::__ANON__ main::__ANON__ (xsub)
0000s0smain::::RUNTIME main::RUNTIME
Call graph for these subroutines as a Graphviz dot language file.
Line State
ments		 Time
on line		 Calls Time
in subs		 Code
02422µsProfile data that couldn't be associated with a specific line:
# spent 315µs making 1 call to Text::CSV_XS::BEGIN@1240 # spent 107µs making 1 call to File::Temp::END
111.81ms#!/usr/bin/env perl
2
3229µs117µs
# spent 17µs within main::BEGIN@3 which was called: # once (17µs+0s) by main::NULL at line 3
use v5.36;
# spent 17µs making 1 call to main::BEGIN@3
4289µs22.06ms
# spent 2.06ms (164µs+1.89) within main::BEGIN@4 which was called: # once (164µs+1.89ms) by main::NULL at line 4
use utf8;
# spent 2.06ms making 1 call to main::BEGIN@4 # spent 1µs making 1 call to utf8::import
5374µs31.72ms
# spent 1.27ms (770µs+496µs) within main::BEGIN@5 which was called: # once (770µs+496µs) by main::NULL at line 5
use strictures 2; # nice `use strict`, `use warnings` defaults
# spent 1.27ms making 1 call to main::BEGIN@5 # spent 436µs making 1 call to strictures::import # spent 16µs making 1 call to strictures::VERSION
62113µs2704µs
# spent 673µs (625+48) within main::BEGIN@6 which was called: # once (625µs+48µs) by main::NULL at line 6
use open qw(:utf8); # try to use Perl's internal Unicode encoding for everything
# spent 673µs making 1 call to main::BEGIN@6 # spent 31µs making 1 call to open::import
7117µs12µs
# spent 2µs within main::BEGIN@7 which was called: # once (2µs+0s) by main::NULL at line 7
BEGIN{$diagnostics::PRETTY = 1} # a bit noisy, but somewhat informative
# spent 2µs making 1 call to main::BEGIN@7
82819µs2218ms
# spent 218ms (99.4+119) within main::BEGIN@8 which was called: # once (99.4ms+119ms) by main::NULL at line 8
use diagnostics -verbose;
# spent 218ms making 1 call to main::BEGIN@8 # spent 21µs making 1 call to diagnostics::import
9
10# Carp
112116µs259µs
# spent 34µs (8+25) within main::BEGIN@11 which was called: # once (8µs+25µs) by main::NULL at line 11
use Carp;
# spent 34µs making 1 call to main::BEGIN@11 # spent 25µs making 1 call to Exporter::import
12285µs22.46ms
# spent 2.39ms (669µs+1.72) within main::BEGIN@12 which was called: # once (669µs+1.72ms) by main::NULL at line 12
use Carp::Assert;
# spent 2.39ms making 1 call to main::BEGIN@12 # spent 66µs making 1 call to Carp::Assert::import
13# filepath functions
14283µs22.39ms
# spent 2.37ms (2.16+208µs) within main::BEGIN@14 which was called: # once (2.16ms+208µs) by main::NULL at line 14
use Cwd qw(abs_path);
# spent 2.37ms making 1 call to main::BEGIN@14 # spent 24µs making 1 call to Exporter::import
152117µs21.03ms
# spent 1.00ms (896µs+108µs) within main::BEGIN@15 which was called: # once (896µs+108µs) by main::NULL at line 15
use File::Basename qw(basename dirname);
# spent 1.00ms making 1 call to main::BEGIN@15 # spent 22µs making 1 call to Exporter::import
16287µs22.57ms
# spent 2.57ms (1.46+1.11) within main::BEGIN@16 which was called: # once (1.46ms+1.11ms) by main::NULL at line 16
use File::Spec;
# spent 2.57ms making 1 call to main::BEGIN@16 # spent 700ns making 1 call to main::__ANON__
17# misc file utilities
182152µs213.8ms
# spent 13.8ms (4.66+9.10) within main::BEGIN@18 which was called: # once (4.66ms+9.10ms) by main::NULL at line 18
use File::Temp;
# spent 13.8ms making 1 call to main::BEGIN@18 # spent 28µs making 1 call to Exporter::import
19276µs27.16ms
# spent 7.14ms (1.64+5.50) within main::BEGIN@19 which was called: # once (1.64ms+5.50ms) by main::NULL at line 19
use File::Slurp;
# spent 7.14ms making 1 call to main::BEGIN@19 # spent 23µs making 1 call to Exporter::import
20276µs27.15ms
# spent 7.14ms (508µs+6.63) within main::BEGIN@20 which was called: # once (508µs+6.63ms) by main::NULL at line 20
use Text::CSV;
# spent 7.14ms making 1 call to main::BEGIN@20 # spent 5µs making 1 call to Exporter::import
21273µs24.80ms
# spent 4.73ms (1.22+3.51) within main::BEGIN@21 which was called: # once (1.22ms+3.51ms) by main::NULL at line 21
use JSON;
# spent 4.73ms making 1 call to main::BEGIN@21 # spent 68µs making 1 call to JSON::import
22270µs22.24ms
# spent 2.22ms (832µs+1.39) within main::BEGIN@22 which was called: # once (832µs+1.39ms) by main::NULL at line 22
use YAML::XS;
# spent 2.22ms making 1 call to main::BEGIN@22 # spent 15µs making 1 call to Exporter::import
23# misc scripting IO utilities
24286µs219.1ms
# spent 19.1ms (7.70+11.4) within main::BEGIN@24 which was called: # once (7.70ms+11.4ms) by main::NULL at line 24
use IO::Prompter;
# spent 19.1ms making 1 call to main::BEGIN@24 # spent 14µs making 1 call to IO::Prompter::import
25 # `capture_stdout` for backticks w/o shell (escaping issues)
26283µs23.17ms
# spent 3.10ms (1.94+1.16) within main::BEGIN@26 which was called: # once (1.94ms+1.16ms) by main::NULL at line 26
use Capture::Tiny qw(:all);
# spent 3.10ms making 1 call to main::BEGIN@26 # spent 75µs making 1 call to Exporter::import
27 # for more complicated stuff
28 # eg timeout, redirection
29280µs214.2ms
# spent 14.2ms (11.8+2.35) within main::BEGIN@29 which was called: # once (11.8ms+2.35ms) by main::NULL at line 29
use IPC::Run qw(run);
# spent 14.2ms making 1 call to main::BEGIN@29 # spent 12µs making 1 call to Exporter::import
302114µs228.3ms
# spent 28.3ms (4.78+23.6) within main::BEGIN@30 which was called: # once (4.78ms+23.6ms) by main::NULL at line 30
use IPC::Cmd qw(can_run);
# spent 28.3ms making 1 call to main::BEGIN@30 # spent 18µs making 1 call to Exporter::import
31# option/arg handling
322118µs26.09ms
# spent 6.00ms (5.42+578µs) within main::BEGIN@32 which was called: # once (5.42ms+578µs) by main::NULL at line 32
use Getopt::Long qw(:config gnu_getopt auto_version); # auto_help not the greatest
# spent 6.00ms making 1 call to main::BEGIN@32 # spent 88µs making 1 call to Getopt::Long::import
332110µs227.2ms
# spent 27.1ms (1.66+25.5) within main::BEGIN@33 which was called: # once (1.66ms+25.5ms) by main::NULL at line 33
use Pod::Usage;
# spent 27.1ms making 1 call to main::BEGIN@33 # spent 62µs making 1 call to Exporter::import
34# use local modules
35
# spent 598µs (410+188) within main::BEGIN@35 which was called: # once (410µs+188µs) by main::NULL at line 38
use lib (
36150µs7161µs dirname(abs_path($0)),
# spent 52µs making 2 calls to File::Basename::dirname, avg 26µs/call # spent 44µs making 1 call to lib::import # spent 35µs making 3 calls to Cwd::abs_path, avg 12µs/call # spent 30µs making 1 call to File::Spec::Unix::rel2abs
37 abs_path(File::Spec->rel2abs('../lib/', dirname(abs_path($0)))),
381110µs1598µs ); # https://stackoverflow.com/a/46550384
# spent 598µs making 1 call to main::BEGIN@35
39
40# turn on features
412103µs2156µs
# spent 151µs (106+45) within main::BEGIN@41 which was called: # once (106µs+45µs) by main::NULL at line 41
use builtin qw(true false is_bool reftype);
# spent 151µs making 1 call to main::BEGIN@41 # spent 5µs making 1 call to builtin::import
42215µs229µs
# spent 16µs (4+13) within main::BEGIN@42 which was called: # once (4µs+13µs) by main::NULL at line 42
no warnings 'experimental::builtin';
# spent 16µs making 1 call to main::BEGIN@42 # spent 13µs making 1 call to warnings::unimport
432844µs2928µs
# spent 894µs (860+34) within main::BEGIN@43 which was called: # once (860µs+34µs) by main::NULL at line 43
use feature 'try';
# spent 894µs making 1 call to main::BEGIN@43 # spent 34µs making 1 call to feature::import
44227µs220µs
# spent 12µs (4+8) within main::BEGIN@44 which was called: # once (4µs+8µs) by main::NULL at line 44
no warnings 'experimental::try';
# spent 12µs making 1 call to main::BEGIN@44 # spent 8µs making 1 call to warnings::unimport
45
46111µs14µs our $VERSION = version->declare('v2022.12.30');
# spent 4µs making 1 call to version::declare
47# end prelude
48285µs214.9ms
# spent 14.7ms (1.03+13.7) within main::BEGIN@48 which was called: # once (1.03ms+13.7ms) by main::NULL at line 48
use Data::Printer;
# spent 14.7ms making 1 call to main::BEGIN@48 # spent 247µs making 1 call to Data::Printer::import
49
50290µs21.28ms
# spent 1.28ms (470µs+807µs) within main::BEGIN@50 which was called: # once (470µs+807µs) by main::NULL at line 50
use Gradescope::Translate;
# spent 1.28ms making 1 call to main::BEGIN@50 # spent 6µs making 1 call to Exporter::import
512694µs260.6ms
# spent 60.6ms (404µs+60.2) within main::BEGIN@51 which was called: # once (404µs+60.2ms) by main::NULL at line 51
use Gradescope::Color qw(color_print);
# spent 60.6ms making 1 call to main::BEGIN@51 # spent 24µs making 1 call to Exporter::import
52
531200nsmy %options;
5412µs14µsGetOptions(\%options,
# spent 4µs making 1 call to Getopt::Long::GetOptions
55 'help|h|?',
56 'delimiter|d=s',
57 'keyheader|k=s@',
58 'valueheader|v=s@',
59 'tokenfilter|t|f|p=s@',
60 ) or pod2usage(-exitval => 1, -verbose => 2);
611500nspod2usage(-exitval => 0, -verbose => 2) if $options{help} || @ARGV < 1;
62
631400ns$options{delimiter} //= ':';
641200ns$options{keyheader} //= ['token'];
651200ns$options{valueheader} //= ['submission'];
661100ns$options{tokenfilter} //= ['true'];
67
681900ns$options{keyheader} = [$options{delimiter}, @{$options{keyheader}}];
69
701400nsmy ($submissions) = @ARGV;
71
721500nsmy $token2uniqname = do {
7312µs local $/ = undef;
74118µs275µs JSON::from_json <STDIN>;
# spent 62µs making 1 call to JSON::from_json # spent 13µs making 1 call to main::CORE:readline
75};
76120µsmy %token2uniqname = %{$token2uniqname};
77
781100nsmy %submissions;
791205µsfor my $token (keys %token2uniqname){
80 my %filtered = Gradescope::Translate::read_csv($submissions,
81 $options{keyheader}, $options{valueheader},
82
# spent 53701s (5.84+53695) within main::__ANON__[split.pl:90] which was called 133035 times, avg 404ms/call: # 133035 times (5.84s+53695s) by Text::CSV_XS::__ANON__[/home/hejohns/perl5/lib/perl5/x86_64-linux-gnu-thread-multi/Text/CSV_XS.pm:1460] at line 1457 of Text/CSV_XS.pm, avg 404ms/call
sub { # see Text::CSV for details
8313303552.5ms my $pred = false;
84 #say JSON::to_json $_[1];
85133035138ms my @row = @{$_[1]}; # capture_stdout takes a code ref, so $_[1] inside is shadowed
86
# spent 53563s (1.46+53561) within main::__ANON__[split.pl:88] which was called 133035 times, avg 403ms/call: # 133035 times (1.46s+53561s) by Capture::Tiny::_capture_tee at line 382 of Capture/Tiny.pm, avg 403ms/call
capture_stdout {
871330352.06s26607053561s $pred = run [@{$options{tokenfilter}}, $token], '<', \(JSON::to_json \@row);
# spent 53558s making 133035 calls to IPC::Run::run, avg 403ms/call # spent 3.64s making 133035 calls to JSON::to_json, avg 27µs/call
881330353.00s133035487ms };
# spent 487ms making 133035 calls to Capture::Tiny::capture_stdout, avg 4µs/call
891330351.37s return $pred;
90 },
911052.69ms10553715s );
# spent 53715s making 105 calls to Gradescope::Translate::read_csv, avg 512s/call
92105341µs delete @filtered{grep {!defined $filtered{$_}} keys %filtered};
93105206µs next if keys %filtered == 0; # some students may not have submissions
9484424µs $submissions{$token} = \%filtered;
95}
96
971790µs2849mscolor_print(JSON::to_json(\%submissions, {pretty => 1, canonical => 1}), 'JSON');
# spent 831ms making 1 call to Gradescope::Color::color_print # spent 18.6ms making 1 call to JSON::to_json
98
99# PODNAME:
100# ABSTRACT: Gradescope submission script component
101=pod
102
103=encoding utf8
104
105=head1 SYNOPSIS
106
107split.pl : () -> json
108
109split.pl [options] I<submissions>
110
111split.pl [-t ./field-n-eq?.pl] [-d ':' -k problem_id -v score] submissions.csv < token2uniqname.json
112
113=head1 DESCRIPTION
114
115splits up the I<submissions> csv
116into individual chunks for upload
117
118=head1 OPTIONS
119
120=head2 help|h|?
121
122=head2 delimiter|d
123
124=head2 keyheader|k
125
126=head2 valueheader|v
127
128keyheader and valueheader will be passed to perl's C<Text::CSV>
129to convert I<submissions> to a key-value, for each token2uniqname token
130(ie stdout is a key-value, keyed by token, value is another key-value)
131
132this may require joining multiple csv columns for the key,
133so a delimiter may be specified.
134
135=head2 tokenfilter|t
136
137command will be fed
138a csv row as json through stdin,
139and passed a student's token as a last argument
140
141command is a predicate that should C<exit 0> iff the csv row should be used for the student with that token
142
143=head3 bundled lambdas
144
145=over 4
146
147=item F<./field-n-eq?.pl>
148
149=back
150
151=cut
152
- -
 
# spent 17µs within Internals::SvREADONLY which was called 41 times, avg 405ns/call: # 39 times (16µs+0s) by constant::import at line 164 of constant.pm, avg 413ns/call # once (400ns+0s) by constant::BEGIN@24 at line 33 of constant.pm # once (100ns+0s) by constant::BEGIN@24 at line 34 of constant.pm
sub Internals::SvREADONLY; # xsub
# spent 4µs within PerlIO::Layer::find which was called 3 times, avg 1µs/call: # 3 times (4µs+0s) by open::import at line 88 of open.pm, avg 1µs/call
sub PerlIO::Layer::find; # xsub
# spent 51µs within UNIVERSAL::VERSION which was called 12 times, avg 4µs/call: # once (8µs+0s) by File::Temp::BEGIN@149 at line 149 of File/Temp.pm # once (6µs+0s) by File::Temp::BEGIN@176 at line 176 of File/Temp.pm # once (6µs+0s) by strictures::VERSION at line 99 of strictures.pm # once (5µs+0s) by File::Temp::BEGIN@167 at line 167 of File/Temp.pm # once (5µs+0s) by File::Temp::BEGIN@150 at line 150 of File/Temp.pm # once (5µs+0s) by Text::CSV::BEGIN@1 at line 1 of (eval 11)[Text/CSV.pm:114] # once (4µs+0s) by File::Temp::BEGIN@147 at line 147 of File/Temp.pm # once (4µs+0s) by JSON::BEGIN@2 at line 2 of (eval 12)[JSON.pm:274] # once (3µs+0s) by File::Temp::BEGIN@153 at line 24 of Scalar/Util.pm # once (3µs+0s) by JSON::Backend::XS::init at line 25 of (eval 13)[JSON.pm:295] # once (900ns+0s) by JSON::Backend::XS::init at line 27 of (eval 13)[JSON.pm:295] # once (900ns+0s) by main::BEGIN@5 at line 105 of strictures.pm
sub UNIVERSAL::VERSION; # xsub
# spent 12µs within UNIVERSAL::can which was called 10 times, avg 1µs/call: # 3 times (3µs+0s) by if::work at line 16 of if.pm, avg 1µs/call # 3 times (2µs+0s) by strictures::_enable at line 130 of strictures.pm, avg 833ns/call # once (3µs+0s) by attributes::import at line 63 of attributes.pm # once (2µs+0s) by main::BEGIN@30 at line 1611 of IPC/Cmd.pm # once (1µs+0s) by Capture::Tiny::BEGIN@14 at line 16 of Capture/Tiny.pm # once (1µs+0s) by ExtUtils::MakeMaker::BEGIN@11 at line 26 of ExtUtils/MakeMaker/version.pm
sub UNIVERSAL::can; # xsub
# spent 2.48s within UNIVERSAL::isa which was called 1729578 times, avg 1µs/call: # 399108 times (1.52s+0s) by IPC::Run::harness at line 1800 of IPC/Run.pm, avg 4µs/call # 399108 times (120ms+0s) by IPC::Run::_open_pipes at line 2152 of IPC/Run.pm, avg 302ns/call # 266072 times (98.8ms+0s) by IPC::Run::IO::_new_internal at line 175 of IPC/Run/IO.pm, avg 371ns/call # 133036 times (274ms+0s) by IPC::Run::_clobber at line 2960 of IPC/Run.pm, avg 2µs/call # 133036 times (274ms+0s) by IPC::Run::_cleanup at line 3265 of IPC/Run.pm, avg 2µs/call # 133036 times (99.0ms+0s) by IPC::Run::IO::_init_filters at line 490 of IPC/Run/IO.pm, avg 744ns/call # 133036 times (59.6ms+0s) by IPC::Run::_spawn at line 1440 of IPC/Run.pm, avg 448ns/call # 133036 times (37.4ms+0s) by IPC::Run::harness at line 1916 of IPC/Run.pm, avg 281ns/call # 105 times (130µs+0s) by Text::CSV_XS::error_diag at line 687 of Text/CSV_XS.pm, avg 1µs/call # 3 times (3µs+0s) by base::import at line 94 of base.pm, avg 1µs/call # once (700ns+0s) by Getopt::Long::GetOptionsFromArray at line 333 of Getopt/Long.pm # once (600ns+0s) by Getopt::Long::GetOptionsFromArray at line 475 of Getopt/Long.pm
sub UNIVERSAL::isa; # xsub
# spent 2µs within main::CORE:pack which was called: # once (2µs+0s) by IPC::Cmd::BEGIN@5 at line 196 of IPC/Open3.pm
sub main::CORE:pack; # opcode
# spent 13µs within main::CORE:readline which was called: # once (13µs+0s) by main::RUNTIME at line 74
sub main::CORE:readline; # opcode
# spent 700ns within main::__ANON__ which was called: # once (700ns+0s) by main::BEGIN@16 at line 16
sub main::__ANON__; # xsub
# spent 18µs within mro::method_changed_in which was called 40 times, avg 437ns/call: # 40 times (18µs+0s) by constant::import at line 198 of constant.pm, avg 437ns/call
sub mro::method_changed_in; # xsub
Report produced by the NYTProf 6.12 Perl profiler, developed by Tim Bunce based on work by Adam Kaplan and Salvador Fandiño García.