documentation updates; fix for null commands with redirection expansion errors; changes to job notifications for interactive shells sourcing files; fix underflow issue with word_top

COMPAT

@@ -432,6 +432,10 @@ version and versions 2.0 and above.
     option processing as bindable command names for which to print any key
     bindings.
 
+69. Interactive shells will notify the user of completed jobs while sourcing
+    a script. Newer versions defer notification until script execution
+    completes.
+
 Shell Compatibility Level
 =========================
 
@@ -589,7 +593,9 @@ compat52 (set using BASH_COMPAT)
 	  expressions composed of five or more primaries.
 	- the -p and -P options to the bind builtin treat remaining arguments
 	  as bindable command names for which to print any key bindings
-
+	- interactive shells will notify the user of completed jobs while
+	  sourcing a script. Newer versions defer notification until script
+	  execution completes.
 -------------------------------------------------------------------------------
 
 Copying and distribution of this file, with or without modification,

CWRU/CWRU.chlog

@@ -10070,3 +10070,43 @@ lib/readline/doc/rluser.texi,lib/readline/doc/readline.3,doc/bash.1
 	- Readline Introduction: expand description of the Meta key and the
 	  various things it can do, reference force-meta-prefix and
 	  enable-meta-key
+
+				   8/26
+				   ----
+doc/bashref.texi
+	- case: add process substitution to the list of expansions for the
+	  word.
+	  Reported by shynur <one.last.kiss@outlook.com>
+
+execute_cmd.c
+	- execute_null_command: if an empty simple command (no words, just
+	  redirections) requires a fork, reset top_level and exit if we
+	  longjmp there. This can happen with an expansion error while we
+	  evaluate the redirection.
+	  Report from youheng.lue@gmail.com
+	- execute_connection: in default mode, bash performs jobs notifications
+	  in an interactive shell between commands separated by ';' or '\n'.
+	  It shouldn't do this in posix mode, since posix now specifies when
+	  notifications can take place
+
+examples/loadables/{realpath,csv,dsv,cut,stat,kv}
+	- changes to make them build when arrays are not available
+	  Report from Dennis Clarke <dclarke@blastwave.org>
+
+jobs.c
+	- notify_and_cleanup: make interactive shells notifying during sourced
+	  scripts dependent on the shell compatibility level and inactive in
+	  versions beyond bash-5.2
+	  Inspired by report from Zachary Santer <zsanter@gmail.com>
+
+doc/bashref.texi
+	- Bash posix mode: note the change with job notifications in command
+	  lists
+	- Shell compatibility mode: note job notification changes for compat
+	  level > 52
+
+parse.y
+	- CHECK_FOR_RESERVED_WORD: call set_word_top here on the reserved word
+	  token about to be returned; don't worry about trying to set it after
+	  the fact
+	  Report from Collin Funk <collin.funk1@gmail.com>

MANIFEST

@@ -1160,6 +1160,7 @@ tests/errors8.sub	f
 tests/errors9.sub	f
 tests/errors10.sub	f
 tests/errors11.sub	f
+tests/errors12.sub	f
 tests/execscript	f
 tests/exec.right	f
 tests/exec1.sub		f	755

builtins/read.def

@@ -201,6 +201,7 @@ read_builtin_timeout (int fd)
 #endif
 }
 
+#if defined (READLINE)
 void
 reset_rl_instream (FILE *save_instream)
 {
@@ -213,6 +214,7 @@ uw_reset_rl_instream (void *fp)
 {
   reset_rl_instream (fp);
 }
+#endif
 
 /* Read the value of the shell variables whose names follow.
    The reading is done from the current input stream, whatever

doc/bash.1

@@ -1412,7 +1412,7 @@ only be referenced; assignment to them is not allowed.
 Expands to the positional parameters, starting from one.
 When the expansion is not within double quotes, each positional parameter
 expands to a separate word.
-In contexts where it is performed, those words
+In contexts where these expansions are performed, those words
 are subject to further word splitting and pathname expansion.
 When the expansion occurs within double quotes, it expands to a single word
 with the value of each parameter separated by the first character of the

doc/bashref.texi

@@ -1041,9 +1041,9 @@ as a @var{clause}.
 
 Each clause must be terminated with @samp{;;}, @samp{;&}, or @samp{;;&}.
 The @var{word} undergoes tilde expansion, parameter expansion, command
-substitution, arithmetic expansion, and quote removal
+substitution, process substitution, arithmetic expansion, and quote removal
 (@pxref{Shell Parameter Expansion})
-before matching is attempted.
+before the shell attempts to match the pattern.
 Each @var{pattern} undergoes tilde expansion, parameter expansion,
 command substitution, arithmetic expansion, process substitution, and
 quote removal.
@@ -1833,7 +1833,7 @@ only be referenced; assignment to them is not allowed.
 ($*) Expands to the positional parameters, starting from one.
 When the expansion is not within double quotes, each positional parameter
 expands to a separate word.
-In contexts where it is performed, those words
+In contexts where these expansions are performed, those words
 are subject to further word splitting and filename expansion.
 When the expansion occurs within double quotes, it expands to a single word
 with the value of each parameter separated by the first character of the
@@ -8544,6 +8544,10 @@ The message printed by the job control code and builtins when a job
 is stopped is `Stopped(@var{signame})', where @var{signame} is, for
 example, @code{SIGTSTP}.
 
+@item
+Bash does not perform job notifications between executing commands in
+lists separated by @samp{;} or newline in interactive shells.
+
 @item
 Alias expansion is always enabled, even in non-interactive shells.
 
@@ -9118,6 +9122,10 @@ If the @option{-p} or @option{-P} option is supplied to the @code{bind} builtin,
 as bindable command names, and
 displays any key sequences bound to those commands, instead of treating
 the arguments as key sequences to bind.
+@item
+Interactive shells will notify the user of completed jobs while sourcing a
+script.
+Newer versions defer notification until script execution completes.
 @end itemize
 
 @end table

doc/version.texi

@@ -2,11 +2,10 @@
 Copyright (C) 1988-2024 Free Software Foundation, Inc.
 @end ignore
 
-@set LASTCHANGE Fri Aug 16 17:10:06 EDT 2024
-
+@set LASTCHANGE Mon Aug 26 11:29:52 EDT 2024
 
 @set EDITION 5.3
 @set VERSION 5.3
 
-@set UPDATED 16 August 2024
+@set UPDATED 26 August 2024
 @set UPDATED-MONTH August 2024

examples/loadables/csv.c

@@ -30,6 +30,8 @@
 
 #include "loadables.h"
 
+#if defined (ARRAY_VARS)
+
 #define CSV_ARRAY_DEFAULT	"CSV"
 
 #define NQUOTE	0
@@ -99,11 +101,13 @@ csvsplit (SHELL_VAR *csv, char *line, char *dstring)
 
   return (rval = ind);				/* number of fields */
 }
+#endif
 
 int
 csv_builtin (WORD_LIST *list)
 {
   int opt, rval;
+#if defined (ARRAY_VARS)
   char *array_name, *csvstring;
   SHELL_VAR *v;
 
@@ -164,6 +168,10 @@ csv_builtin (WORD_LIST *list)
 
   opt = csvsplit (v, csvstring, ",");
   /* Maybe do something with OPT here, it's the number of fields */
+#else
+  builtin_error ("arrays not available");
+  rval = EXECUTION_FAILURE;
+#endif
 
   return (rval);
 }

examples/loadables/cut.c

@@ -178,7 +178,9 @@ cutbytes (SHELL_VAR *v, char *line, struct cutop *ops)
   if (v)
     {
       ind = 0;
+#if defined (ARRAY_VARS)
       bind_array_element (v, ind, buf, 0);
+#endif
       ind++;
     }
   else
@@ -246,7 +248,9 @@ cutchars (SHELL_VAR *v, char *line, struct cutop *ops)
   if (v)
     {
       ind = 0;
+#if defined (ARRAY_VARS)
       bind_array_element (v, ind, buf, 0);
+#endif
       ind++;
     }
   else
@@ -301,7 +305,9 @@ cutfields (SHELL_VAR *v, char *line, struct cutop *ops)
 	return ind;
       if (v)
 	{
+#if defined (ARRAY_VARS)
 	  bind_array_element (v, ind, line, 0);
+#endif
 	  ind++;
 	}
       else
@@ -330,12 +336,14 @@ cutfields (SHELL_VAR *v, char *line, struct cutop *ops)
     {
       if (bmap[b] == 0)
 	continue;
+#if defined (ARRAY_VARS)
       if (v)
 	{
 	  bind_array_element (v, ind, fields[b], 0);
 	  ind++;
 	}
       else
+#endif
 	{
 	  if (i == 0)
 	    putchar (ops->delim);
@@ -441,8 +449,13 @@ cut_internal (int which, WORD_LIST *list)
       switch (opt)
 	{
 	case 'a':
+#if defined (ARRAY_VARS)
 	  array_name = list_optarg;
 	  break;
+#else
+	  builtin_error ("arrays not available");
+	  return (EX_USAGE);
+#endif
 	case 'b':
 	  cutflags |= BFLAG;
 	  list_arg = list_optarg;
@@ -502,6 +515,7 @@ cut_internal (int which, WORD_LIST *list)
       return (EXECUTION_FAILURE);
     }
 
+#if defined (ARRAY_VARS)
   if (array_name)
     {
       v = builtin_find_indexed_array (array_name, 1);
@@ -511,6 +525,7 @@ cut_internal (int which, WORD_LIST *list)
 	  return (EXECUTION_FAILURE);
 	}
     }
+#endif
 
   op.flags = cutflags;
   op.delim = delim;

examples/loadables/dsv.c

@@ -30,6 +30,8 @@
 
 #include "loadables.h"
 
+#if defined (ARRAY_VARS)
+
 #define DSV_ARRAY_DEFAULT	"DSV"
 
 #define NQUOTE	0
@@ -158,11 +160,13 @@ dsvsplit (SHELL_VAR *dsv, char *line, char *dstring, int flags)
 
   return (rval = ind);				/* number of fields */
 }
+#endif
 
 int
 dsv_builtin (WORD_LIST *list)
 {
   int opt, rval, flags;
+#if defined (ARRAY_VARS)
   char *array_name, *dsvstring, *delims;
   SHELL_VAR *v;
 
@@ -238,6 +242,10 @@ dsv_builtin (WORD_LIST *list)
 
   opt = dsvsplit (v, dsvstring, delims, flags);
   /* Maybe do something with OPT here, it's the number of fields */
+#else
+  builtin_error ("arrays not available");
+  rval = EXECUTION_FAILURE;
+#endif
 
   return (rval);
 }

examples/loadables/kv.c

@@ -37,6 +37,8 @@
 extern int errno;
 #endif
 
+#if defined (ARRAY_VARS)
+
 #define KV_ARRAY_DEFAULT	"KV"
 
 /* Split LINE into a key and value, with the delimiter between the key and
@@ -113,10 +115,12 @@ kvfile (SHELL_VAR *v, int fd, char *delims, char *rs)
   QUIT;
   return nr;  
 }
+#endif
 
 int
 kv_builtin (WORD_LIST *list)
 {
+#if defined (ARRAY_VARS)
   int opt, rval, free_delims;
   char *array_name, *delims, *rs;
   SHELL_VAR *v;
@@ -192,6 +196,10 @@ kv_builtin (WORD_LIST *list)
   if (free_delims)
     free (delims);	/* getifs returns allocated memory */  
   return (rval > 0 ? EXECUTION_SUCCESS : EXECUTION_FAILURE);
+#else
+  builtin_error ("arrays not available");
+  return (EXECUTION_FAILURE);
+#endif
 }
 
 /* Called when builtin is enabled and loaded from the shared object.  If this

examples/loadables/realpath.c

@@ -165,10 +165,12 @@ realpath_builtin(WORD_LIST *list)
 				builtin_error("%s: %s", p, strerror(errno));
 			continue;
 		}
+#if defined (ARRAY_VARS)
 		if (aflag) {
 			bind_array_element (v, ind, r, 0);
 			ind++;
 		}
+#endif
 		if (qflag == 0) {
 			if (vflag)
 				printf ("%s -> ", p);

examples/loadables/stat.c

@@ -46,6 +46,8 @@
 extern int	errno;
 #endif
 
+#if defined (ARRAY_VARS)
+
 #define ST_NAME		0
 #define ST_DEV		1
 #define ST_INO		2
@@ -334,10 +336,12 @@ loadstat (char *vname, SHELL_VAR *var, char *fname, int flags, char *fmt, struct
     }
   return 0;
 }
+#endif
 
 int
 stat_builtin (WORD_LIST *list)
 {
+#if defined (ARRAY_VARS)
   int opt, flags;
   char *aname, *fname, *timefmt;
   struct stat st;
@@ -410,6 +414,10 @@ stat_builtin (WORD_LIST *list)
     }
 
   return (EXECUTION_SUCCESS);
+#else
+  builtin_error ("arrays not available");
+  return (EXECUTION_FAILURE);
+#endif
 }
 
 /* An array of strings forming the `long' documentation for a builtin xxx,

execute_cmd.c

@@ -2862,7 +2862,7 @@ execute_connection (COMMAND *command, int asynchronous, int pipe_in, int pipe_ou
 
       QUIT;
 #if defined (JOB_CONTROL)
-      if (command->value.Connection->connector == ';' && job_control && interactive)
+      if (command->value.Connection->connector == ';' && job_control && interactive && posixly_correct == 0)
         notify_and_cleanup ();
 #endif
       optimize_connection_fork (command);			/* XXX */
@@ -4187,7 +4187,7 @@ bind_lastarg (char *arg)
 static int
 execute_null_command (REDIRECT *redirects, int pipe_in, int pipe_out, int async)
 {
-  int r;
+  int r, code;
   int forcefork, fork_flags;
   REDIRECT *rd;
 
@@ -4222,6 +4222,10 @@ execute_null_command (REDIRECT *redirects, int pipe_in, int pipe_out, int async)
 	  if (pipe_in != NO_PIPE || pipe_out != NO_PIPE)
 	    subshell_environment |= SUBSHELL_PIPE;
 
+	  code = setjmp_nosigs (top_level);
+	  if (code)
+	    exit (EXECUTION_FAILURE);
+
 	  if (do_redirections (redirects, RX_ACTIVE) == 0)
 	    exit (EXECUTION_SUCCESS);
 	  else

jobs.c

@@ -3614,8 +3614,10 @@ notify_and_cleanup (void)
   if (jobs_list_frozen > 0)
     return;
 
-  if (want_job_notifications || interactive || interactive_shell == 0 || sourcelevel)
+  if (want_job_notifications || interactive || interactive_shell == 0)
     notify_of_job_status ();
+  else if (interactive_shell && sourcelevel && shell_compatibility_level <= 52)
+    notify_of_job_status ();	/* XXX - was not dependent on BASH_COMPAT */
 
   if (jobs_list_frozen < 0)
     return;		/* status changes only */

parse.y

@@ -3137,8 +3137,7 @@ static int open_brace_count;
 	      else if (word_token_alist[i].token == '}' && open_brace_count) \
 		open_brace_count--; \
 \
-	      if (last_read_token == IF || last_read_token == WHILE || last_read_token == UNTIL) \
-		set_word_top (last_read_token); \
+	      set_word_top (word_token_alist[i].token); \
 \
 	      if (posixly_correct) \
 		parser_state &= ~PST_ALEXPNEXT; \
@@ -3596,9 +3595,6 @@ read_token (int command)
       parser_state &= ~PST_ASSIGNOK;
       parser_state &= ~PST_CMDBLTIN;
 
-      if (last_read_token == IF || last_read_token == WHILE || last_read_token == UNTIL)
-	set_word_top (last_read_token);
-
       return (character);
     }
 
@@ -3717,6 +3713,7 @@ read_token (int command)
       /* case pattern lists may be preceded by an optional left paren.  If
 	 we're not trying to parse a case pattern list, the left paren
 	 indicates a subshell. */
+      /* XXX - check for last_read_token != WORD before setting PST_SUBSHELL? */
       if MBTEST(character == '(' && (parser_state & PST_CASEPAT) == 0) /* ) */
 	parser_state |= PST_SUBSHELL;
       /*(*/
@@ -4843,7 +4840,6 @@ parse_dparen (int c)
 #if defined (ARITH_FOR_COMMAND)
   if (last_read_token == FOR)
     {
-      set_word_top (last_read_token);
       arith_for_lineno = line_number;
       cmdtyp = parse_arith_cmd (&wval, 0);
       if (cmdtyp == 1)
@@ -4863,8 +4859,6 @@ parse_dparen (int c)
     {
       sline = line_number;
 
-      if (last_read_token == IF || last_read_token == WHILE || last_read_token == UNTIL)
-	set_word_top (last_read_token);
       cmdtyp = parse_arith_cmd (&wval, 0);
       if (cmdtyp == 1)	/* arithmetic command */
 	{
@@ -5792,7 +5786,6 @@ got_token:
       expecting_in_token++;
       break;
     }
-  set_word_top (last_read_token);
 
   return (result);
 }

tests/errors.right

@@ -330,6 +330,11 @@ bash: line 1: readonly: `AA[4]': not a valid identifier
 array: 1
 sh: line 1: export: `AA[4]': not a valid identifier
 sh: line 1: readonly: `AA[4]': not a valid identifier
+bash: -c: line 5: syntax error: unexpected end of file from command on line 1
+bash: -c: line 3: syntax error: unexpected end of file from command on line 1
+bash: -c: line 4: syntax error: unexpected end of file from command on line 1
+bash: -c: line 5: syntax error: unexpected end of file from command on line 1
+bash: -c: line 7: syntax error: unexpected end of file from command on line 1
 bash: line 1: return: can only `return' from a function or sourced script
 after return
 bash: line 1: return: can only `return' from a function or sourced script
@@ -338,4 +343,4 @@ sh: line 1: unset: `a-b': not a valid identifier
 sh: line 1: /nosuchfile: No such file or directory
 sh: line 1: trap: SIGNOSIG: invalid signal specification
 after trap
-./errors.tests: line 393: `!!': not a valid identifier
+./errors.tests: line 396: `!!': not a valid identifier

tests/errors.tests

@@ -371,6 +371,9 @@ ${THIS_SH} ./errors10.sub
 # invalid identifiers to readonly/export
 ${THIS_SH} ./errors11.sub
 
+# EOF when parsing compound commands
+${THIS_SH} ./errors12.sub
+
 ${THIS_SH} -c 'return ; echo after return' bash
 ${THIS_SH} -o posix -c 'return ; echo after return' bash
 

tests/errors12.sub

@@ -0,0 +1,42 @@
+: ${THIS_SH:=./bash}
+
+# various error messages about incomplete compound commands
+
+${THIS_SH} -c 'if
+	true; true
+then
+	echo foo bar' bash
+
+${THIS_SH} -c 'while false; do
+	echo true' bash
+
+${THIS_SH} -c 'until false
+do
+	echo false
+' bash
+
+${THIS_SH} -c 'for f
+in 1 2 3
+do
+	: ; :' bash
+
+${THIS_SH} -c 'case foo in
+bar)	if false
+	then
+		true
+	fi
+	;;
+' bash
+
+# this tripped up ubsan
+x()
+{
+    case y in         
+        z)    
+            if (! false); then
+              foo=bar
+            fi
+            ;;
+    esac
+}
+